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(57) Abstract 

A video data stream analyzer (300) modifies an input digital video signal so that the resulting output digital signal can be optimally 
compressed by a digital video encoder (103). The video data stream analyzer eliminates redundancy in the input video signal, and reorganizes 
the input video signal so that the spatial and temporal redundancy is increased, and generates side channel information to tell the video 
encoder whether vertical frame-based filtering or vertical field-based filtering is preferable, and specifies the order and duration of the display 
of the fields (123, 130) after decoding (121). The video data stream analyzer provides scan direction detection of the incoming video digital 
data, automatically detects scene cuts, repeated fields, and mixed-field frames, drops repeated fields, and replaces a frame with a scene cut 
with a frame having, identical fields for video, cartoon, or telecine, or a combination. 
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An Analyzer and Methods for Detecting 
and Processing Video Data Types 
in a Video Data Stream 



Field of the Invention 

This invention relates generally to compression of 
video data and more specifically to real time detection of 
each of a variety of common video data streams and real 
10 time modification of the video data streams. 

Background of the Invention 

Video data typically includes luminance and 
chrominance data for each pixel in a frame. Raw digital 
video data contains too much information for transmittal 

15 over normal communication media and requires considerable 
storage capacity. Consequently, to utilize either the 
limited bandwidth of the communication media, or storage 
capacity efficiently, coding techniques are commonly used 
to compress the information contained in raw digital video. 

2 0 data. 

For example, in Figure 1, video camera 101 generates 
an analog video signal that drives input processor 102 in 
encoding system 110. Input processor 102 digitizes and 
typically filters the analog video signal to produce a raw 

25 digital video signal. The raw digital video signal is 
encoded, i.e., compressed, by encoder 103. 

The compressed digital video signal is transmitted 
over a - communications channel, for example, a satellite 
link, to a decoding system 12 0 that includes a 

30 decoder 121, a post-processor 122, and a display 

driver 123 . Decoder 121 decompresses the encoded video 
data and supplies the resulting signal to post- 
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processor 122, which in turn smooths and enhances the 
video signal. The video signal from post -processor l 22 
suppl.es display driver 123 that drives display unit 130 

The encoding, i.e., compression, of video signals for 
storage or transmission and the subsequent decoding is 
well-known. Moreover, the effectiveness of the 
compression is increased if a priori information 
concerning the content of the raw digital video data is 
available and exploited. 

10 and ' mP ° rtant faCt ° r in the efficiency of the encoding 

and decoding processes is the prediction efficiency. Most 
commonly used encoding processes employ a motion 
compensated prediction loop, a motion compensated 

is Lt Cti ! n l0 ° P " included < for ^ the H.261, 

■ 15 MPEGl and „ PEG2 video compression standards. See for 
example, ITU-T Recommendation H.261, "Codec for 
Audiovisual Services at px64 Kbps, - Geneva 1 993 ; x SO 
11172-1 (MPEGl) , -coding of Moving Pictures and Associated 
Audio for D ig ital storage Med±a gfc up ^ ^ ± _ 

20 Part I,™ 1993; and CD ISO/IEC 13818-2 (MPEG2 ) , .-Generic 
Codang of Moving Pictures and Associated Audio, ■' i 993 
which are each incorporated herein by reference in its 
entirety. 

^Typically, in motion compensated prediction, a small 
25 portion of the current frame, usually a 16 pixel by 16 
P«el block, is compared with a set of similarly sized 
blocks taken from a previously encoded frame, called the 
;: . .reference frame, which is stored in encoder 103 . m the 

encoding process, a difference metric, such as 
30 mean-squared or mean-absolute difference between the 
current block and a reference block, is used for a 
comparison of the blocks. The block in the reference 
frame that best matches. the current block, i.e., has the 
smallest difference metric, is chosen as the prediction 
35 block for the current block. 
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After the prediction block is selected, the 
difference between the current block and the prediction 
block, or in some cases, between the current block and a 
weighted prediction block, is computed to form a 
5 difference block. The difference block is then encoded 
and transmitted. Simultaneously, a vector pointing to the 
location of the prediction block in the reference frame is 
also transmitted as side channel information. 

Decoder 121 has a copy of the reference frame in its 
10 memory. On receiving the prediction block vector, 

decoder 121 fetches the prediction block from the memory, 
and adds the prediction block to the decoded difference 
block to generate the new decoded block. 

A motion compensated prediction loop is effective in 
15 increasing the encoding efficiency because most of the 

time there is a good correlation between successive frames 
of a sequence. Clearly, the better the prediction is, the 
better the encoding efficiency is, because a better 
prediction means less information in the difference block 
20 and so less information to encode and transmit. 

The content of the raw digital data depends on 
several factors. Video signals of commercial interest 
have a variety of formats. Typically, the format of the 
video signal is determined by the source of the video 
25 data. For example, the video frames supplied to input 
processor 102 may represent the output of a vidicon 
adhering to the NTSC prescribed rate of 3 0 frames per 
second (fps) , or the video frames may represent certain 
cartoon sequences that are produced at 15 fps and then 
30 converted to 30 fps by repeating each frame once. (Herein, 
the NTSC prescribed rate is taken as 30 fps, which is 60 
fields per second. As is known by those skilled in the 
art, the actual NTSC prescribed rate is 29.97 fps, which 
is 59.94 fields per second.) Herein, a frame includes two 
35 fields, an odd parity field and an even parity field. The 
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first field contains lines l throuah * ^ 

-Lypxcal display monitors disolav 
followed by the second field fQr °"^! y the f " St field 

5 data stream Alco ! ! the video 

cream. Also, each video source 101 is said «-„ 
produce a type of video data. t0 

If the characteristics of the video h=*- = 
be reliably detected in rea l time tL h ^ 
could be exploited to increase The ^ • CharaCtSristics 
10 compression. For exam^e th efflcien ^ the data 

seated frames gene^e I ' i ^s ~ f ° <~ ^ 

converted to the NTSC r st -» ^ * n P ' 

telecine », ■ 6 ° f ields /second by a 

^-^'^ * — - - - 

temporally disjoint A fill 7 frame are 

J A Ellln sequence, in contr«h 

^ progressive which means that all the : 7 contra st, is 

' frame., is captured at the 7 ^formation in a 

=f<-u«q at the same time in^i-ani- »-i 
are some cases when the o instant. Also, there 

A 3:2 pulldown increases the frame rate of 
sequence by periodi«n, a mov ie 

film segued £ an fields in the 

quence. As an example, consider the 24 f„- 
sequence given in Figure 2A T h» i / P fllm 

35 denote successive frames .n'the "* * C " " ' " 

meS xn the sequence while the 
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numbers "1" and "2" denote the odd and even parity fieids, 
respectively, in each frame. In a general situation, 
frames a b c ... can be assumed to be different . The 
telecine machine increases the frame rate to 3 0 fps by 
5 replacing the set of frames in Figure 2A with the set of 
frames in Figure 2B. 

The sequence of frames in Figure 2B has a five -frame 
periodicity. Each set of four frames of the original 
24 fps sequence (Figure 2A) is converted to five frames by 
10 repeating once the first and sixth fields of the four 
frame set. Moreover, the repetition is done in a manner 
that introduces two mixed- frames in the five frame set. 
In Figure 2B, the mixed- frames are underlined. 
Mixed- field frames are those frames that contain fields 
15 from different temporal instants and are introduced in 
this case by the aforementioned frame rate conversion 
process. Herein, the term mixed-field frame and the term 
mixed- frame are used interchangeably. 

From the point of encoder 103 such a 3:2 pulldown 
20 operation is doubly inefficient. First, part of the 

information is coded redundantly because of repetition of 
some fields. Secondly, forty percent of the frames are 
now "mixed" (alb2, blc2, elf2 and flg2 here) , which means 
they are composed of temporally disjoint fields. This 
25 introduces interlace effects, which in turn result in a 
drop in the prediction efficiency of the frame-based 
encoding. For example, in a motion compensated prediction 
loop, if a mixed- frame is suddenly introduced into the 
sequence, the mixed- frame decreases significantly the 
30 probability of finding a good prediction block. As a 
result, the best possible difference block has a 
significant amount. of information that must be encoded and 
transmitted. 

Hence, appearance of mixed- field frames in the input 
3 5 video "data sequence affects the compression performance of 
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encoder 103 significantly. Thus , it woula be beneficial 
to remove or, if possible, repair such frames before 
compression. Such processing could be performed in either 
real or non-real time. Unfortunately, the inventors are 
unaware of any prior art method or apparatus for removing 
or repairing such mixed- frames in real time. 

A significant additional savings in data rate 
transmission could be achieved by eliminating transmission 
ot one, all, or most repeated fields. Similarly 
10 elimination of mixed-field frames would improve the 
prediction mechanism employed by commonly used video 
compression schemes. 

There are other instances when mixed- field frames 
might be present in a video sequence. Sometimes mixed- 

ield frames are intentionally created in the studio, but 
often mixed-field frames are accidentally introduced 
during editing operations. For example, in Figure 2C a 
normal 30 fps interlaced video sequence is represented. 
An edit was intended to switch from a first scene to a new 
20 scene beginning with frame uXu 2 , but. in the editing 

mZTl' ' "J*"" 1 -" <^ was created (Figure 2D). 

Mixed-field frame dlu2 is said to contain a scene cut 
Similarly, in Figure 2E, two telecine sequences were 

>5 £wV" d merged ' edlting PrOCSSS seated mixed- 

-5 field frame clv2 that contains a scene cut. 

Thus, the mixed-field frames in Figures 2D and 2E 
were created by mid-frame edits. m general, mid-frame 
edits refer to an abrupt change in scene in the middle of 
a frame, i.e., the odd and even parity fields of a frame 
0 belong to different scenes. Such frames are also 
difficult to compress efficiently. 

To meet a wide variety of user requirements, and 
bandwidth constraints, video compressors need to process 
_ video frames of different spatial resolutions, even if the 
> original source material was all of the same resolution 
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Spatial resolution is altered by digitally filtering each 
frame of sequence before the frame is compressed. The 
filtering may be done in the horizontal direction, the 
vertical direction, or both. When filtering is done in 
5 the vertical direction, preferably the nature of the 
frame, i.e., whether the frame is interlaced or 
progressive, would be taken into account, because 
different modes of vertical filtering are required for 
interlaced and progressive frames to produce good visual 
10 quality in decoding system 120. Typically, interlaced 
frames have a better visual quality when the frames are 
vertically filtered on a field basis, while the opposite 
is true for progressive frames. 

SUMMARY OF THE INVENTION 

15 . According to the principles of this invention, a 
video data stream analyzer modifies, preferably in real 
time, an input digital video signal. The video data 
stream analyzer of this invention eliminates redundancy in 
the input video signal, and reorganizes the input video 

2 0 signal so that the spatial and temporal redundancy is 

increased. Hence, the modifications to the video signal 
made by the video data stream analyzer of this invention 
increase the efficiency of encoding the video signal . 

In addition, the video data stream analyzer generates 

25 side channel information that aids in selecting an 
appropriate vertical filter, if required, and in 
reconstructing the encoded data. Specifically, the video 
data stream analyzer generates, in real time, side channel 
information that is supplied to the video encoder. The 

30 side channel information tells the video encoder whether 
vertical frame -based filtering or vertical field-based 
filtering is preferable. Additional side channel 
information specifies the order and duration of the 
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display of the fields after decoding and this inform- - ■ 
Preferably i s encoded with the vide! signal lnf0rmatl ° n 
The video data stream analyzer provides real time 

s ^""r of the incoming video di * itai ^ 

fields ^ reliably det6CtS SCene repeated 

fields and mixed . field frames , n P d 

:^!° d ! h ta ^ rSal time ^^ndent of the viL l ta 
Zla *T Stream analy26r modif i- the xnput 

10 renl * repeated and 

Ident f ;. frame W±th 3 SC6ne CUt With * ^ving 
identical fields for video, cartoon, telecine video dL 

type S we u as arbitrary combinations of thes ; d ;° de d o ata 

datatypes. The prior art video compression 

15 fra e mL f " intr0dUCed * mixed-field 
frames, and repeated fields a re either completely 

anaT;:e a r e o d f°t r h 9reatly * V "~ ^ 

anaiyzer of this invention. 

parameters and couoled i-,-> ^ Plurality of 

3 0 the pluralitv aTT "ordering memory, analyzes 

Plurality of parameters and selects one of the fields 
stored -xa the reordering memory for output at each f ^ 
interval onto the reordering memory outp^ " 

35 out ^^^t^^ 1 *- a - 

*-y and a plurality of differential 
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accumulators connected to the FIFO. The FIFO, in one 
embodiment, includes a first field memory connected to the 
statistical analyzer input bus and a second field memory 
connected to the first field memory and to the statistical 
5 analyzer output bus. The plurality of differential 
accumulators includes: a strip activity differential 
accumulator that has a first input bus connected to the 
statistical analyzer input bus and a second input bus 
connected to the statistical analyzer output bus; a 

10 differential activity differential accumulator having a 
first input bus connected to the statistical analyzer 
output bus and a second input bus connected between the 
first field memory and the second field memory; and an 
intra-activity differential accumulator having a first 

15 input bus connected to the statistical analyzer output bus 
and a second input bus connected to an output bus of a 
line delay memory that has an input bus connected to the 
statistical analyzer output bus. 

The controller in the video data stream analyzer 

20 includes (i) a statistical processor coupled to receive 
the parameters from the statistical analyzer and to a 
plurality of registers, and (ii) a state machine coupled 
to the statistical analyzer, to the reordering memory, to 
an output counter, and to the plurality of registers. The 

25 state of a bit in each of the plurality of registers 

represents a. characteristic of a field in the reordering 
memory. Specifically, one register in the plurality of 
registers is a repeated field register and a bit in the 
repeated field register indicates, in one embodiment, 

3 0 whether a field in the reordering memory is a repeated 
field. One register in the plurality of registers is a . . 
scene cut register and a bit in the scene cut field 
register indicates whether a scene cut occurs at a field 
in the reordering memory. One register in the plurality. 

3 5 of registers is a scan pattern register and a bit in the 
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scan pattern register indicates whether a f ield Dair . 
the reordering memory is progressive or interl^cL o 
register in the plurality n f • menaced. One 

^are register ^ " ° dd — 

5 real timeT* • T^' " ^ -al y2 es in 

real time the information in the pluralitv of / • 
and detects- a i. Plurality of registers 

etects. a scene change in the video data str«™ 
repeated field in the video data stream 71 * 
video sequence in the video data 11 ^ " 

XO sequence; a change from a vi^T* ^ * ^ 

-ream to a telecine "queL a ^ ^ ^ 

sequence in the video daT 1 T ' 3 ^ 
change from a cartoon seance \n thV TV^^ * 
a telecine sequence,- a chlge f^ a .Illle 
» the video data stream to a cartoon sequence an^T ^ 
from a telecine sequence in the video^T'/ 96 
video sequence . 3ta Stream to * 

Further, the operation of the state re- 
combination with the statistical Lo " 
20 between, i e detect StlCal proce "« can distinguish 

d ata st ; eams ; 0 t:: d ::r two different ±nput vid - 
Ltust; otaiiTT r n in ^T^r- 

to video sequences,- and the fTf" polibl" ^ 
sequences . y posslb le movie to movie 

In one embodiment, the <! f = f 0 
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state, and a transfer state. The state machine also 
includes a plurality of status conditions that are derived 
from information in the plurality of registers . The 
status conditions are utilized to determine the transition 
5 between various states in the state machine. 

The state machine remains in the video state so long 
as the input video data stream is a video sequence . The 
state machine transitions from the video state to the 
cartoon state upon detecting a sequence of fields in the 

10 input video data stream that have characteristics 
indicating a possible start of a cartoon sequence. 

The state machine transitions from the video state to 
the first seek state upon the state machine detecting a 
sequence of fields in the input video data stream that 

15 indicates that a telecine sequence may be starting. The 
state machine also transitions from the cartoon state to 
the first seek state under certain conditions . 

The state machine transitions from the first seek 
state to the movie state upon detection of a telecine 

20 sequence. The state machine transitions from the movie 
state to a second seek state to determine whether the 
telecine sequence continues in the input video data 
stream. So long as the telecine sequence continues, the 
state machine oscillates between the movie state and the 

25 second seek state. When the second seek state detects 
that the telecine sequence in not continuing, the state 
machine transfers to a transfer state that controls 
termination of the telecine sequence. 

Each state in the state machine has a plurality of 

30 sub-states. Some sub-states simply transition to the next 
sub-state, while other sub-states generate control 
information for the reordering memory and side channel 
information if the output counter has a predetermined 
value. The side channel information includes a display 

3 5 code and an output format code. The control information 
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th,t f« . p , rtlcul , r tr „_ no t e 

output bus nr, - ° ut ' uc bu * 

"up': :r th A vi r —- *. 

25 ^ 9eneratiD9 3 P^ality of parameters that 

1 ■ ■ * PlUrali ^ of Parameters to identify a 

■ video data type in the plurality of v±deo 

for each fieid in the stream of video data. 
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encoding efficiency of the video data stream. Preferably, 
the video data stream analyzer performs the method in real 
time so that the output data stream from the video data 
steam analyzer is passed directly to a video encoder. 
5 Herein, real time means that the video data stream 

analyzer receives input video data at a predetermined rate 
and processes each field without delaying the input video 
data stream and without danger of overwriting or otherwise 
losing any portion of the incoming video data . 

10 Numerous methods are incorporated in the video data 

stream analyzer of this invention. The video data stream 
analyzer analyzes the input video data stream to determine 
whether each field pair in the video data steam has a 
progressive or an interlaced format . This input scan 

15 detection is done in a scan pattern analysis process 

within the controller. The scan pattern analysis process 
is performed for each field pair in the input video data 
stream. In an index generation step, a plurality of scan 
pattern activity indices for a field pair are generated by 

20 the controller, using parameters from the statistical 

analyzer, i.e., the characteristics of each field pair in 
the input video data stream are monitored. In one 
embodiment, each scan pattern activity index is an inter- 
intra index for each strip. Following the index 

2 5 generation step, each of the scan pattern activity indices 

are compared to a scan pattern threshold in an interlaced- 
progressive analysis step. If a predetermined number of 
the scan pattern activity indices are greater than the 
scan pattern threshold, the field pair is identified as 

3 0 interlaced and otherwise as progressive. 

The video data stream analyzer also includes a method 
for processing telecine and cartoon sequences in a video 
data stream to enhance the efficiency of encoding the 
video data stream. The method includes the steps of: 
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to d "° n ; t0ring SaCh field - "» video data stream 
to detect repeated fields wherein upon detection of a 
repeated field, the repeated field is flagged- 

analyzing a pattern formed by the fields flagged 
as repeated to identify whether the repeated fielS 
are m a cartoon sequence or a telecine sequence • 
dropping repeated fields from the video data 
stream based upon identification of the repeated 
field as being in one of the cartoon sequence and the 
ceiecme sequence. 



tl St, : T"' ^ ^^ ^ ^ P~*««d by 
tl •'■"•t±«l Processor while 

the stat e machine performs the analysis and dropping 

15 ST" Preferabl - a " - th. steps are perfo^d L real 

The video data stream analyzer detects, i e 
identifies, a scene cut in a video data stream byi 

measuring a cumulative difference in activity 
dataT 3 f±rSt fiSld 3 Parlty in vldj 

video data stream; and 

identifying , scene cut between the first and 
second fxelds upon the cumulative difference- 
satisfying a predetermined condition. 

: £ Tl^^T*' CUmUlatiVe in activity 

Plurality of r^' ^ fiSld aCtlVity 13 thS "» ^ I 
coZll P activities. The predetermined 

.. condxtxon includes comparison of the cumulative different 

so t a ::; v rL with a scene cut thresh ° id - *~ — 

30 threshold xs, in one case, a running average of the field 
activities for the fields of the given parity The e 
cut threshold is updated after detection^ of e'ach Ine 
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cut. The scene cut threshold is updated only if another 
predetermined condition is true, i.e, there is not a 
sudden deviation between the field activity of the field 
and the scene cut threshold. In another embodiment, the 
5 predetermined condition includes comparison of the 

cumulative difference in activity with a minimum value 
threshold and a scene cut threshold. 

The video data stream analyzer also performs a method 
for detecting repeated fields in the input video data 
10 stream. In one embodiment, the method for detecting 
repeated fields includes : 



determining if a primary criterion is true for a 
first field in a frame; and 

identifying the first field as an unrepeated 
field upon determining that the primary criterion is 
not true . 



In one embodiment, the method next determines if the 
primary criterion is true for the other field in the frame 
upon determining that the primary criterion is true for 

20 the first field. The first field is identified as a 

repeated field upon determining that the primary criterion 
is not true for the other field in the frame. If the 
primary criterion is true for the other field in the 
frame, the method determines if a secondary criterion is 

25 true for the first field. If the secondary criterion is 
not true for the first field, the first field is not a 
repeated field. If the secondary criterion is true, for 
the first field, the method determines if a secondary 
criterion is true for the other field in the frame. If 

3 0 the secondary criterion is not true for the other field in 
the frame, the first field is identified as a repeated 
field. 
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in an alternative embodiment, a method for detecting 
a repeated field in a video data stream having a p lural 
of frames wherein each frame includes a field of a f irst 
parity and a field of a second parity, the met hod 
5 comprises the steps of: 

comparing a field activity of a field with a 
first threshold; and 

identifying the field as a repeated field if the 
field activity of the field is less than the first 
threshold and the field has the second parity. 

If the field does not have the second parity, the method 
compares a strip activity of each strip of the 
field with a second threshold; and 

identifies the field as a repeated field if a 
predetermined number of the strip activities of the 
field are less than the second threshold and the 
field has the second parity. 

Next, the method 

repeats the field activity and strip activity 
comparison steps for a second field of the second 

^ (i) the field has the first **^y> tii) 

the field activity of the field is less than the 
first threshold; and (iii) a predetermined number of 
t the. strip activities of the field are less than the 
second threshold; and 

identifies the field as a repeated field if <i, 
the f.eld activity of the second field is greater 
than the first threshold; or (ii) a predetermined 
number of the strip activities of the second field 
are greater than the second threshold. 
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Finally, the method 

repeats the field activity and strip activity 
comparison steps for a second field of the second 
parity if (i) the field has the first parity; (ii) 
5 the field activity of the field is less than the 

first threshold; and (iii) a predetermined number of 
the strip activities of the field are less than the 
threshold; 

compares a measure of field activity of the 
10 field with a third threshold if (i) the field 

activity of the second field is less than the first 
threshold; and (ii) a predetermined number of the 
strip activities of the second field are less than 
the second threshold; and 
15 identifies the field as a repeated field if the 

measure of field activity of the field is less than 
the third threshold. 

Hence, a plurality of conditions can be used, according to 
the principles of this invention to detect repeated fields 
20 in a video data stream. Moreover, each condition can be 
used individually, or in a selected combination with other 
conditions . 

A real time method for detecting a telecine sequence 
in a sequence of frames of video data wherein each frame 
25 has a field of a first parity followed by ; a field of a 
second parity, according to the principles of this 
invention, includes the steps of: 



30 



monitoring, in real time, each field in the 
video data stream to detect repeated fields and 
flagging each repeated field; and 
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analyzing, in real time, a pattern formed by the 
fields flagged as repeated to identify whether the 
repeated fields are in a telecine sequence. 

5 f^ t n ly ii n3 StSP lnClUdeS determi -»* tether every 
5 fifth field is a repeated field. 

A method for detecting a cartoon sequence in a 

S fZTZ °V rames of video data wherein each *~ a 

field of a first parity followed by a field of a second 
parity includes the steps of: second 

10 • monitoring each field in the sequence to detect 

repeated fields and flagging each repeated field; and 
analyzing a pattern formed by the fields flagged 
as repeated to identify whether the repeated field! 
are m a cartoon sequence. 

15 f^irf 2 ' 119 StSP ±nClUdeS dete --^ "^ther the two 
fields in every other frame of the sequence are flagged as 
repeated fields. Preferable nagged as 

Y ' the m °nitoring and analvzino 
steps are performed in real time. .analyzing 

20 methodT^ 119 ^ PrinCi ^ les of this invention, a 
method for improving encoding efficiency of a video data 
stream includes the steps of: 

analyzing each field in the video data stream to 
detect repeated fields; and 

videoTr 1119 e3Ch deteCtSd rSPeated f ield fr ™ ^e 
video data stream to generate a- modified video data 

stream thereby eliminating encoding of redundant 

information and so improving encoding efficiency. 

The method further includes the step of: 
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generating a display code for each frame in the 
modified video data stream wherein the display code 
indicates a display combination for fields in the 
modified video data stream to recreate the video data 
5 stream. 

Another real time method for improving encoding 
efficiency of a video data stream includes the steps of : 

analyzing each frame in the video data stream to 
detect frames that include a mid- frame edit; 
10 dropping a first field in a frame in which a 

mid- frame edit was detected; and 

replacing the first field with a second field in 
the frame in which a mid- frame edit was detected 
thereby eliminating an abrupt change in activity 
15 within the frame and so improving encoding 

efficiency. 

A method for introducing dead times in a video data 
stream to a video encoder includes the steps of : 

storing for each frame in the video data stream 
2 0 a field of a first parity and a field of a second 

parity; 

passing selected stored fields to the video 
encoder in response to a first output field selection 
code; and 

25 passing no stored fields to the video encoder in 

response to a second output field selection code 
wherein the second output field selection code is 
used to introduce a dead time in the video data 
stream. 



10 
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The methods and apparatus of this invention operate 
on a v.deo data stream and produce a modified video lata 
stre am and side channel information, The modified v^ 
data stream „ organized to improve the efficiency of 
5 encoding the video data stream. Preferably, the methods 
and apparatus of this invention operate in real time 
However, the methods and apparatus can also be utilized in 
non-real time applications. utilized m 

BRIEF PFSCRIPTTOM n P THE np W T», M 

6nr H F±SrUre 1 iS a block dia 9"m of . a prior art video 
encoding and decoding systems. 

Figure 2A is a representation of a normal 24 frames 
per second progressive movie sequence 

15 per sllZT " " * repreSentati ° n «* * normal 24 fr.ames 
per second progressive movie sequence converted to a 30 

^:; s Per SSCOnd —gh a telecine 

Figure 2C is a representation of a normal 3 0 frames 
per second interlaced video sequence 

Figure 2D is . representation of two interlaced video 

nc :: at r fo K r a first scene ^ - *°* • — 

scene that have been combined so that mixed- frame, i e 
one with a scene cut . between fields in a frame, is 

created. . ' ° 

Figure 2E is a representation of two telecine 
sequences, one for a first scene and one for a second 
.scene that have been combined so that a mixed-frame i . 
ye wxth a, scene cut between fields in a frame, is 
created. 

Figure 3 illustrates the location of the video data 
stream analyzer of this invention within the prior art 
encoding system. F rc 

vide/nT 6 ^ " 3 bl ° Ck dia9ram ° f ° ne «bodl-nt of the 
video data stream analyzer of this invention. 
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Figure 4B is a more detailed block diagram of one 
embodiment of the video data stream analyzer of this 
invention. 

Figure 5A is a first state diagram for the state 
5 machine within the controller of Figures 4A and 4B. 

Figure SB is a second state diagram for the state 
machine within the controller of Figures 4A and 4B. 

Figures 6A to 6C are illustrations of alternative 
ways to insert dead time intervals in the output data 
10 stream of the video data stream analyzer of this 
invention. 

Figure 7A is a process flow diagram for one 
embodiment of the process performed by the video data 
stream analyzer of this invention. 
15 Figure 7B is a process flow diagram for one 

embodiment of the scan detection process of this 
invention. 

Figure 7C is a more detailed process flow diagram for 
selected steps in the scan detection process of Figure 7B. 
20 Figure 7D is a more detailed of one embodiment of the 

steps 703 and 704 in process 700 of Figure 7A. 

Figure 8A is a process flow diagram for one 
embodiment of the scene cut detection process of this 
invention . 

25 Figure 8B is a process flow diagram for one 

embodiment of the updating of the scene cut threshold for 
odd parity fields. 

Figure 8C is a process flow diagram for one 
embodiment of the updating of the scene cut threshold for 
3 0 even parity fields. 

Figure 9 A is a process flow diagram for one 
embodiment of the real time repeated field detection 
process of this invention. 
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Figures 9B and 9C are a process flow diagram for 
another embodiment of the real time repeat field detection 
as applied to a cartoon sequence and a telecine sequence 
Figure 10 is a state diagram for the video state 
5 within the state machine within the controller of Figures 
4A and 4B. 

Figure 11 is a state diagram for the first seek state 
within the state machine within the controller of Figures 
4A and 4B. 

10 Figure 12 is a state diagram for the movie state 

within the state machine within the controller of Figures 
4A and 4B. 

Figure 13 is a state diagram for the second seek 
= tate "ithin the state machine within the controller of 
15 Figures 4A and 4B . 

Figure 14 is a state diagram for the transfer state 
within the state machine within the controller of Figures 
4A and 4B. 

,„ .^. Fi9UrS 15 13 3 State ^ram for the cartoon state 
20 within the state machine within the controller of Figures 
4A and 4B . - 

Figure 16 is an even more detailed block diagram for 
the statistics analyzer of this invention. 

25 ^ /if" " 3 SChematic "Presentation for each of 

25 the differential accumulators in Figures 4B and 16 

Figure 17B illustrates, the timing characteristics of 
.signals used in the video data stream analyzer of this 

invention 1 .' 

V Figure 18 is a more detailed block diagram of the 
30 reordering memory of this invention. 

Figure 19 is a schematic diagram of the two banks of 
video random access memory. in the reordering memory of 
tins invention. 

« „„ Fi9Ure 20 ±S 3 Schematic diagram of the read-write 
3 5 address generation circuit of this invention. 
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Figure 21 is a state diagram for a state machine 
within the VRAM controller of Figure 18. 

Figure 22 is a timing diagram for transfer of 
information from the DRAM port to the serial port of the 
5 VRAMs forming the reordering memory. 

Figure 23 is a timing diagram for a refresh cycle of 
the VRAM in the reordering memory. 

Figure 24 is a timing diagram for a 128 word page 
mode write cycle to the VRAM in the reordering memory. 
10 Figure 25 is a timing diagram for a transfer from the 

serial port to the output line of the video data stream 
analyzer of this invention. 

DETAILED DESCRIPTION OF THE INVENTION 

According to the principles of this invention, a 

15 video data stream analyzer modifies, preferably in real 
time, an input digital video signal so that the resulting 
output digital signal can be optimally compressed, in one 
embodiment, using a prior art digital video encoder. The 
video data stream analyzer of this invention eliminates 

20 redundancy, e.g., removes repeated fields, in the input 
video signal, and reorganizes the input video signal so 
that the spatial and temporal redundancy is increased, 
e.g., fixes mid-frame edits and detects and fixes scene 
cuts . 

25 The modifications to the video signal made by the 

video data stream analyzer of this invention increase the 
efficiency of subsequently encoding the video signal. In 
addition, the video data stream analyzer generates, in 
real time, side channel information that aids in selecting 

30 an appropriate vertical filter, if required, and .in 

appropriately displaying the video data after decoding. 
Specifically, as described more completely below, the 
video data stream analyzer generates side channel 
information that is supplied to the video signal encoder. 
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The side channel information tells the video encoder 
whether vertical frame-based filtering or vertical field- 
based faltering is preferable. Additional side channel 
information specifies the order and duration of the 

5 ttTflW ' i£ldS 3fter deC ° ding ^ this -formation 

preferably is encoded with the video signal. 

The video data stream analyzer provides real time 
scan detection of the incoming video digital data, and 

io fW tiCa H ly ^ rSliably detects scene cuts < 

10 fields, and mixed-field frames in the incoming digital 
video data, which is sometimes referred to as "an input 
video data stream," in real time, 

The video data stream analyzer modifies the input 
video data stream by dropping repeated fields and 

15 replacing a frame with a scene cut with a frame having 
identical fields. Hence, the prior art compression 
inefficiencies introduced by scene cuts, mixed-field 
frames and repeated fields are either completely 
eliminated or greatly reduced. 

20 to * S . eXplained more completely below, the modifications 
to the input video data stream are either lossless or 

IZZ'« l f m ° difiCations «« lossless, the modified- 

video data stream from the video data stream analyzer can 
bemused to re-generate the original video data in its 

vST; ° n ° ther hand ' " ' thS «*«i«t±on to the 

video data stream by the video data stream analyzer is 

exactly ° ri9inal V±de ° • data Can not reproduced 

L ,2* Vid6 ° Stream anal y 2er 300 of this invention 

30 (Figure 3, is incorporated in prior art digital video 
encoding system 310 in Pig. 3 between input 
preprocessor 102 and encoder !03. Hence, video data 
stream analyzer 300 receives as an input signal 301 the 
video data stream from input processor 102 and provides a 
35 modified video data stream, i.e., an output video datT 
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stream 302, as an input signal to encoder 103. As 
described more completely below, video data stream 
analyzer 300 also generates information on a side channel 
(not shown) to encoder 103 . The other components in 
5 Figure 3 are identical to the components with the same 

reference numeral in Figure 1 and the description of those 
components is incorporated herein by reference. 

Input signal 301 from input processor 102 is a stream 
of video fields that are accompanied by a field 
10 synchronization signal, sometimes called a VSync signal, 
or simply a field sync. A frame is defined as an adjacent 
odd-even pair of fields, i.e., a field of a first parity 
and a field of a second parity. 

Each field is accompanied by a signal indicating the 
15 parity of the field, i.e., whether the field is an even 
parity field or an odd parity field. Since as defined 
herein, a frame always begins with an odd parity field, 
this signal is used to define a frame boundary and so is 
referred to herein as the frame synchronization signal. 

2 0 Input signal 3 01 may or may not include chrominance 

data. In one embodiment of this invention, the luminance 
data is processed and then identical operations are 
performed on the chrominance data. Thus, any re -ordering 
of the fields by video data stream analyzer 300 is applied 
25 in an identical fashion to the chrominance data. Of 
course, if additional robustness is desired, the 
chrominance data could be independently processed by video 
data stream analyzer 3 00. 

Video data stream analyzer 300 of this invention 

3 0 includes a plurality of features for enhancing the quality 

of the display in decoding system 120. Specifically, 
video data stream analyzer 300 automatically detects via a 
scan detection process if the incoming video data is 
interlaced or progressive and appropriately organizes the 
3 5 data for subsequent vertical resampling, if any. 
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„it-h If K thS inC ° min9 VidS ° data is « interlaced sequence 
with substantia! motion, vertical filtering on a fral 
grid can cause a significant loss in temporal resolution 

5 TZl th6 t tWO fleldS ° f ±nterlaCed f ™ « 

5 at deferent time instants. To minimize such a temporal 

resolution l OSS/ video data stream ^ 
xn information on the side channel that an interlaced 
sequence with substantial motion should be fil tere d on a 
Held gnd instead of a frame grid. 
10 on the other hand, in the absence of substantial 

motion in the interlaced sequence, or in the case of 
progressively scanned data, vertical filtering on the 
frame grid is preferable because such filtering renders 
15 s h c lghe V Patial ««*»t±on. Hence, for progressively 
15 scanned xnput data and interlaced sequences without 
substantial motion, video data stream analyzer 300 
specifies in the information on the side channel that the 
video data should be filtered on B ■ • J . 

field grid. ^ 9rid lnStead of a 

20 one T!' V±de0 StrSam anal ^ 2er 300 organizes, in 

one embodiment, the data in output signal 302 differently 
depending on whether input signal 30! is interlaced For 
an interlaced input signal, output signal 302 is specified 

25 fielf S - T ™ Uml .-» fields » ith odd and eve! parity 

25 fields alternating. For a progressive input signal 

output signal 302 is specified in the side channel as . 
. frames, i.e, with the odd and even parity f ie i ds of a 

frame interleaved. This designation in the side channel 

30 f L r t ° UtPUt Slgnal iS WS11 SUlted vertical 

tvoi!^ 9 re " Samplin ^ if —ded. As mentioned above, 

typically it is preferable to separate the fields of an 
interlaced frame before doing any vertical filtering 
whereas it is preferable to keep the fields of a 

35 sZ r r; iVe frame interleaved - In -other embodiment, the 
35 scan detection process is not utilized by video data 
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stream analyzer 3 00 or alternatively, the scan detection 
process is switched on and off. 

In addition to scan detection, video data stream 
analyzer 300 detects and reverses the effects of the 3:2 
5 pulldown of the telecine operation in real time. 

Specifically, the repeated fields in a telecine sequence 
in input video data stream 3 01 are detected and dropped 
from output video data stream 302. The repeated fields in 
the telecine sequence are dropped independent of whether 

10 the telecine sequence is interlaced or progressive. 
Consequently, output video data stream 3 02 typically 
includes only conventional frames such as those 
illustrated in Figures 2A and 2C. However, to prevent 
buffer management problems in decoding system 12 0, video 

15 data stream analyzer 3 00 introduces, as explained more 

completely below, dead time intervals in output video data 
stream 302, i.e., time intervals in which no video data is 
output in output video data stream 3 02. 

In reality, the periodicity of a telecine mixed- frame 

20 pattern in input signal 301 is constant for only a short 
duration of time. The loss of periodicity is usually a 
result of different factors such as studio editing of a 
telecine sequence and addition of noise into the video 
signal. Two possible effects of these operations are that 

25 either the telecine repetitive pattern of repeated fields 
disappears (because the new sequence is a regular 3 0 fps 
NTSC sequence) in input signal 301, or the telecine 
repetitive pattern in input signal 3 01 takes on a 
different phase. As explained more completely below, 

30 video data stream analyzer 3 00 tracks, in real time, such 
changes in input signal 301 and corrects all telecine 
mixed- frames that are detected. 

Video data stream analyzer 30 0 also detects and 
repairs in real time a mid-f rame edit created in a 

35 transition from an interlaced video sequence to a 
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progressive telecine movie sequence- f rnm 
telecine movie sequence to an^nterUcIT ^/^^ 
. - a passive telecine 

S ~' — * ™ ~ to 

sequence t7Tt\ «d from a cartoon 

v±2nr t r.r e movie seguence and 

LTtSLt" 300 repairs 

to a second vll 0 Zl lZ\ 7 & ^ to ^ 

« — of the ^JTraTJ'rZZZl tT f ^ ^ 
with the other field ThH<= replaClng the Pepped field 

However, the compression efficiencv of = -f 
ger an abrupt change between t-h<* f^u . 

example, video + +- ype * Por 

-pairs elch o^ 3 °° detSCtS a ° d 

2S ,„ hS P ° ssible mid-frame edits create K 

25 scene cut from a first progressive ^ created by a 
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eliminated. Again, the side channel information generated 
by video data stream analyzer 300 tells decoder 121 the 
display order and duration for each field in the video 
signal so that the repaired frame is properly displayed. 
5 Finally, video data stream analyzer 3 00 detects in 

. real time any sequence in input video data stream 3 01 that 
was originally produced at 15 fps and converted to 3 0 fps 
by repeating each frame exactly once, e.g., some cartoon 
sequences. Video data stream analyzer 300 drops the 

10 redundant fields in output video data stream 302. The 
side channel information generated by video data stream 
analyzer 300 tells decoder 121 the display order and. 
duration for each field in the video signal so that the 
original cartoon sequence can be recreated. 

15 Figure 4A is a more detailed block diagram of video 

data stream analyzer 300. Statistical analyzer 401 
receives input video signal 3 01. As illustrated in 
Figure 4A, input video signal 301 includes a plurality of 
fields al, a2 , bl, b2 that include luminance data and 

20 chrominance data, but, in this embodiment, only the 

luminance data is analyzed by statistical analyzer 401. 

Frame 410 includes a first parity field al and a 
second parity field a2 , while frame 411 includes fields bl 
and b2 of the first and second parity, respectively. . As 

25 explained above, each field al, a2, bl, b2 is accompanied 
by pulse 421, 422, 423, 424 respectively of a signal 420 
called "VSync". Also, each field is associated with a 
field signal 430 that indicates the field's parity, i.e, 
even or odd. Since, in this embodiment of the invention,. 

3 0 each frame begins with an odd parity field, signal 431 . 
indicating an odd parity field also indicates the start of 
a frame. Herein, a frame sync is referring to signal 431, 
for example, that indicates an odd parity field. Most of 
the operations of controller 402 are triggered by signal - 

3 5 VSync. 
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As xnput video data stream 301 is clocked in real 
t-e e g., at 30 fp S , through St a tistical analyzer 4 , to 
reordering m emory 403 via bus 415, statistical 

s ztm 01 r nerates a £or & * ch 

field' ; " allty ° f P — e - characterize the 
fxeld as descrxbed more completely below. ln one 
e^bod the parameters _ a ^ 

sta;Lt 1C ar ram r ers for a strip we teen ^ 

10 ;' atlStlCal ana1 ^ 401 and while parameters for the next 
10 strip are being generated, controller 402 retrieves the 
Parameters for the completed strip f rom statistical 
analyse r 401 overs li nes 417 and ^ ^ 

After a fxeld xs clocked through statistical 
analyzer 401, the field is written into reordering 
15 memory 403, which is video random access memory "Lm> 
ir ~~ — -mory 403 is eighths 

anal ^ Parameters for * "eld from statistical 

.0 ar r y ^\lTt aVailable ^ * M ^ d^tely 

memory u^L^r 6 ^ ^ — ^ 

t-i m Unless otherwise explicitly mentioned all 

time xnstances are measured with respect to the fteld 
entering reordering memory 403. Thus a fi„. 
to an edge of field signal 430 that 
25 after all the data of Se f ield T 

memory 403. Likewise Sntered "ordering 

refers to *n f ! * ^ ( ° r frame boundary) 

, refers to an edge of field signal 430 that follows 
■ :. immediately after all t-**. ^ ws 

"adjacent oL ° f 9 frame ' ^fined as an 

ctajacent odd-even pair of fi^i^c u 

me term "previous field" refers t-^ *k -.^ 

3S reordering memory 403 just bef" e the ^ 

oerore the current field. 
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When all the parameters for a field are available, 
i.e., the field has been loaded in reordering memory 403, 
controller 402 generates in real time any additional 
parameters needed using the stored strip parameters. 
5 Controller 4 02 uses the parameters to analyze the field 
and subsequently analyzes the sequence of fields stored in 
reordering memory 403. Based on the analysis of the 
sequence of fields stored in reordering memory 4 03, 
controller 402 either chooses via control signals on 
10 lines 416 at least one field for output in output video 
data stream 3 02 or determines that at this time no fields 
should be output. Controller 402 also generates side 
channel display codes, which indicate how to reconstruct 
the original video input sequence, and format codes, which 
15 indicate whether the sequence is progressive or interlaced 
for side channel 418. Controller 402 also determines the 
location in reordering memory 4 03 into which the next 
incoming field data is placed. 

Hence, in this embodiment, controller 4 02, which 
20 includes a statistical processor 490 and a state 

machine 480 (Figure 4B) , uses the parameters for scan 
detection, fixing mid-frame edits, detecting scene 
changes, and removing redundant fields generated either by 
the 3:2 telecine process, or by changing a 15 fps cartoon 
25 sequence to 30 fps video sequence. In one embodiment, 
controller 402 is a Motorola 68302 microprocessor 
executing the methods of this invention. A non-real time 
simulation of the operation of video data stream analyzer 
300' was compiled and linked using the C++ compiler "Object 
30 Center" sold by Centerline Software of Cambridge, MA and 
executed on a Sparc Station 2 computer (that is sold. by 
Sun Microsystems, of Mountain View, . CA) running the Unix 
operating system. A real time embodiment could be written 
in the C programming language and compiled and linked 
35 using the Microtec "C" cross-compiler for the 68000 



WO 95/15659 

PCT/US94/13287 



- 32 - 



T2TlT b r th ° n 9 MOt ° r0la 68302 • ^ <*f« eoa.- 
generated by this process is stored as firmware wH-h ■ 
video data stream analyzer 300. fl ™"«e Wlt hxn 

5 i»ci T P3th thr ° Ugh Stati ^ical analyzer 401 

5 includes two field memories 450 451 

in the form of a fi^ ■ *• confi S«»d 

As the luminance ^Z^T^ <™» ■ 

t-v, i. id data of ln P u t signal 301 t>ass@-* 

through memory 450 and memory «1 into bordering 
memory 403, three differences are generated J 
10 differential accumulators 452 4« 

15 consists of 240 ,nH . hat each field 

or 240 active lines; however i-F t-H~ „ ^ 
active "Mr,~« • . er ' 1X the number of 

_ -zr: "rr^r- 

«~ .t rt p, ia «£:^v. « ™"° r ' 
en. ,i,^cr::,:«: b -,-!;: r f« i t: . 

-SA, ia'calculah^ u . ' st reactivity « 

bet:ween^ S ^ V"""" the absolute differences 

^<=j.i ct pixel of the current fiai^ 

30 the same p ixe l of the next fTeXd of ^e !" ' "* 
aH the data for a strip has been prdsseT aTit 
is sent to controller 402 a „. . P oce "«<*. an interrupt 

-ads strip activitTsl L tit , 49 ° 
activity SA . when all the da ta of ^ ^ 

3 5 loaded in reordering current field are 

» reordering memory 103/ statistical prQcessor ^ 
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generates an overall "field activity" FA for the current 
field, i.e., a second parameter, which is the sum of the 
fifteen strip activities that have been read for the 
current field. If input signal 3 01 is noiseless and a 
5 field is repeated, each strip activity SA and field 

activity FA are zero. As explained more completely below, 
the strip activities and field activity FA for a field are 
used by controller 402 in the real time detection of 
repeated fields and scene cuts . Strip activity SA is a 
10 cumulative difference measure for a strip while field 
activity FA is a cumulative difference measure for a 
field. 

Differential accumulator 453 generates a third 
parameter for the field, i.e., a difference measure 

15 between two strips in successive fields of different 

parity. Specifically, a difference measure, "differential 
activity" DA, is calculated by summing the absolute 
differences between a pixel of the current field, e.g., 
field al, and the same pixel of the next field of a 

20 different parity, e.g., field a2 . As. indicated above, 
when all the data for a strip has been processed, the 
interrupt is sent to controller 402, and statistical 
processor 490 also reads and stores differential activity 
DA for that strip. 

25 Since differential activity DA is determined for each 

strip in a field, each field has fifteen values of 
differential activity DA, one for each strip. As 
explained more completely, differential activity DA is 
used in this embodiment to determine whether fields are 

3 0 interlaced or progressive. However, differential activity 
DA could also be used for either scene cut or repeated 
field detection. 

Differential accumulator 454 generates a fourth 
parameter for the field, i.e., a difference measure 

35 between every adjacent even-odd line pair belonging to the 
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same field. Specifically, difference measure "intra 
activity" IA is obtained by summing the absolute 

d T e ^T S b6tWeen PiXSl P3irS belon ^ to adjacent even 
and odd lanes from the same field. Again, this 

5 calculation is done on a strip basis. m other words 

P^el differences are obtained between the following line 

pairs of each strip: (0,1), (2,3), (4,5) , (12 , 13) 

(14,15). Differential accumulator 454 is disabled' i e 

10 l"! T 9enSrate My difference ^ during every'other' 
10 lane time interval so that only a difference measure 
between every adjacent even-odd line pair is generated 

Again, when all the data for a strip has been 
processed and the interrupt is sent to controller 402 
statistical processor 490 also reads and stores intra- 
15 strip activity ia for that strip. The intra-strip 

activities are used with the differential activities in 
the scan detection by controller 402 to distinguish 
between interlaced and progressive input video data 
20 fn As e ^ lained »»« completely below, the parameters 
20 for each field generated by differential accumulators 452 
453, and 454 are sampled at regular real time intervals ' 
and used by controller 402 to determine which field if 
any, M . repeated in input video data stream 301. and at 

25 Tut T" inPUt VidS ° 301 a scene 

cut. m addition, these parameters are used to determine 

the scan pattern (i.e., whether interlaced or not) of each 
field-pair combination in the video data stream 
^Controller 402, which as described above includes a 
statistical data processor 490 and a state machine 480 
30 controls the actual writing of the incoming video aZ' 
stream 301 into a particular location, i.e., f ield , in 
reordering memory 403 and as well as the location or 
locations from which to read the field or fields for the 
next outgoing frame. m selecting the next outgoing 
frame, controller 402 must know in real time whether the 
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frame includes a repeated field, a mixed-frame edit, or a 
scene cut, and whether the frame is interlaced or 
progressive. In addition to selecting the next outgoing 
frame, controller 402 also generates the side channel 
5 information describing the output signal format and a 
display code for the frame. 

As explained more completely below, controller 4 02 
uses* statistical data processor 490 and the parameters 
from statistical analyzer 401 to determine whether the 

10 current field is a repeated field, or a field belonging to 
a new scene, and whether the current field is part of a 
sequence that is interlaced or progressive. To make these 
determinations, statistical data processor 490 uses and 
updates several thresholds. As explained more completely 

15 below, the parameters for the current field are compared 
against the thresholds to determine, in real time, the 
characteristics of the current field. At each frame sync, 
controller 402 records the characteristics of the two 
fields in the current frame in a plurality of status 

20 registers 470, e.g., a scene cut register. SC that holds 
the scene cut history, a repeated field register RF that 
holds the status of primary criterion,- an odd-even compare 
register OEC that holds the status of secondary criterion, 
and a scan pattern register SP that holds the scan pattern 

25 history. The primary and secondary criteria for a field 
are described below. Controller 402 also maintains an 
output counter 460 for indicating when to actually drive a 
frame onto output line 302. While in this embodiment 
registers are used, in view of this disclosure those 

30 skilled in the art will appreciate that the important 

aspect is that the information in the registers is stored 
for future use. Accordingly, storage mechanisms other 
than registers can be used to implement the principles of 
this invention. 
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In this embodiment, state machine 480, as explained 
more completely below, has six states, video state 500 
seek A state 510, movie state 520, seek B state 530, ' 
trans state 540, and cartoon state 550 (Figures 5A ' 
5 and SB) . Each state has one or more sub-states. Herein 
a sub- state is defined as one field period of time that ' 
begins with the field sync preceding the field that is 
about to enter reordering memory 4 03, e.g., field 
sync 420, and ends with field sync 421 that follows after 
10 the field is loaded in reordering memory 403. Also, as 
explained more completely below, the transitions between 
states and the operations within a state, e.g., which 
fields in reordering memory 403 are selected for output 
video data stream 302 during the next sub-states are 
15 determined by status conditions. Each status condition is 
defined using information in status registers 470 See 
Table 1 below. m Figures 5A and SB, if a status 
condition is not given along an arrow from one state to 
another, the arrow without the status condition is a 
20 default selection, i.e, the path indicated by this arrow 
is chosen if all other relevant status conditions are not 
true. 

The default mode of operation of video data stream 
analyzer 300 is defined by state zero, i.e., video 

25 state 500, of state machine 480. In this mode of . 
operation, input signal 3 01 is 3 0 fps and output 
signal 302 is 30 fps with no re-ordering or dropping of 
fields. A frame in input signal 301 with a repeated odd 
parity field and without a repeated even parity field 

3 0 i.e, status condition CO, indicates to controller 402 'that 
a telecine sequence may be beginning. Consequently, state 
machxne 480 moves from video state 500 to state one, i e 
Seek A state 510. "' 

In seek A state 510, controller 402 examines status 
35 registers 470 to determine if the fifth field after the- 
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last repeated odd parity field, which is an even parity- 
field, is repeated, i.e, whether status condition C9 is 
true. If condition C9 is true, i.e., the even parity 
field that is the fifth field after the last repeated odd 
5 parity field is itself a repeated field, state machine 480 
moves from state seek A 510 to state two, i.e., movie 
state 520. 

Upon entering movie state 520, controller 402 has 
detected that input signal 3 01 is a telecine movie 

10 sequence. If the fifth field after the previous repeated 
even parity field, which is now an odd parity field, is a 
repeated field, i.e., status condition CO is true, state 
machine 480 moves from movie state 520 into state 3, i.e, 
seek B state 530. 

15 In seek B state 530, controller 402 examines status 

registers 470 to determine if the fifth field after the 
last repeated odd parity field, which is an even parity 
field, is repeated, i.e, whether status condition C9 is 
true. If condition C9 is true, state machine 480 moves. 

20 from state seek B 530 to movie state 520. So long as 
status conditions CO and C9 are alternately satisfied at 
every fifth field, state machine 480 oscillates between 
states 520 and 530. 

Seek A state 510 and seek B state 53 0 are both 

25 required because the process of leaving the respective 
state is different. When controller 402 is in seek B 
state 530, input signal 301 included at least one repeated 
odd parity field and one repeated even parity field, at. 
the correct relative locations. Therefore, whenever state 

30 machine 480 is in seek state B 530, controller 402 is in 
the middle of processing a telecine sequence. 

Conversely, when controller 402 is in Seek A 
state 510, only a repeated odd parity field has been 
detected, but controller 402 must wait to determine 

35 whether the appropriate repeated even parity field follows 
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for a telecine sequence. Thus, in Seek A state 510, the 
fact that input signal 301 is a telecine sequence is not 
yet established, but the start of telecine sequence is 
possible. if controller 402 does not find a repeated 
5 field at the end of seek A state 510, i.e., status 

condition C9 is false, state machine 480 moves from seek A 
state 510 to video state 500. Thus, implicitly assuming 
that the previous repeated odd parity field was a false 
alarm. 

10 on the other hand, when controller 402 does not find 

a repeated field at the end of seek B state 530 i e 
status condition C9 is not true, controller 4 02 'is exiting 
a telecine sequence, and has to be careful about how the 
last few frames of the telecine sequence are remixed, 
e.g., the last frame of the telecine sequence can be any 
one of »ala2», "alb2», »blc2», "clc2», or «dld2" Thus 
when status condition C9 is not true, state machine 48o' 
moves by default from seek B state 530 to state 4, i e 
trans state 540. "' 
20 In trans state 540, controller 402 essentially 

ascertains which one of the possible five' frames in the 
telecine sequence actually ended the telecine sequence 
and ensures that controller 402 only re -mixes the frame (s) 
appropriately. For example, if the last frame in the 
25- telecine sequence was »ala2", the frame is passed through 
as output signal 3 02. However, if last frame in the 
Racine sequence was frame »blc2", controller 402 repairs 
the last two mixed-frames "alb2» and »blc2» so that the 

3 o rr^: frames are not sent - For &x ^> -n-d-f™*. 

30 »alb 2 .. and ..blc2» are both repaired, e.g. sent, as 
frame "blb2" . 

While controller 402 must exit a telecine sequence 
correctly, controller 402 must also sense the beginning of 
a new telecine sequence, if there is one, as quickly as ■ 
35 possxble. Consequently, controller 402 is constantly 
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looking for the beginning of a new telecine sequence. 
Specifically, controller 402 looks for isolated repeated 
odd parity fields and transitions to seek A state 510 to 
look for a repeated even parity field five fields later. 
5 Hence, the numerous arrows showing transitions to seek A 
state 510 in Figure 5A. 

In video state 500, controller 4 02 checks status 
registers 470 to determine whether both even and odd 
parity fields of the current frame are repeated, i.e., 
10 status condition Cll is true. When status condition Cll 
is true, input signal 301 could possibly be a 15 fps 
cartoon sequence. Consequently, state machine 480 moves 
from video state 500 to state five, i.e., cartoon 
state 550. 

15 Specifically, consider the input cartoon sequence: 

ala2 ala2 blb2 blb2 clc2 clc2 dld2 dld2 . . . 

In video state 500, detection that the first frame "ala2" 
is repeated causes the transition from video state 500 to 
cartoon state 550. However, the confirmation that the 

20 input sequence is in fact a cartoon sequence is obtained 
only when it is determined that the following frame 
H blb2 M , which is different from M ala2," is also repeated. 
If the frame following the second n alSi2" frame is not 
different, most probably the input sequence is a "still" 

25 sequence, i.e., with very little motion. It is best to 
treat a still sequence as video because the compression 
efficiency of a still sequence is anyway quite high. 
Thus, in this case, state machine 48 0 transitions from 
cartoon state 550 back to video state 500. Further, even 

30 if frame "blb2" is different from the second M ala2 M frame, 
but frame n blb2 n is not repeated, controller 402 decides 
that the input sequence is not a cartoon sequence and 
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hence it loops back to video state 500 from cartoon 
state 550. 

As explained more completely below, in each state 
controller 402 selects particular fields from reordering 
5 memory 403 as output signal 302. m addition to selecting 
particular fields for output, state, machine 480 also 
determines the time sequence and order of the fields in 
output video data stream 302. This information is buried 
in the display code for each output frame, which is 
10 available along with each frame as a separate side channel 
signal from video data stream analyzer ,300. 

Figure SB is similar to Figure 5A except status 
condition C9 has been replaced by status condition CI As 
explained more completely, status condition CI is true if 
15 a primary criterion for an even parity field is satisfied 
while status condition C9 is true if both a primary 
criterion and a secondary criterion are true. Hence, when 
state machine 480 operates according to the state diagram 
of Figure 5A, the operation is more robust than when state 
20 machine 480 operates according to the state diagram of 
Figure 5B. However, generally, either state diagram 
provides acceptable real time performance. 

The operation of state machine 480 in combination 
with statistical processor 490 detects and repairs most of 
25 the mixed-frames in the 71 possible edit combinations 

involving video and telecine sequences in input video data 
stream 301. Appendix I gives the input video data 
• stream 301 in row "In"; the output video data stream 302 
xn row "Out"; a display code, as defined in Table 3 below 
30 in row "Display code"; and the video sequence displayed, by 
decoding system 120 using after decoding the output video 
data stream and display code in row "Display." No te that 
the sequences given in Appendix I are not inclusive of all 
xnput video data streams processed by video data stream 
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analyzer 300. Transitions to and from cartoon sequences, 
for example , are not included in Appendix I . 

The input video data streams considered in Appendix I 
represent a pure video input data stream; a video input 
5 data stream that includes a mid- frame edit; the ten 
possible video to movie transitions; the ten possible 
movie to video transitions; and the fifty possible movie 
to movie transitions. The features of video data stream 
analyzer 300, as explained in more detail below, process 
10 each of the various input video data streams given in 

Appendix I and generate the output video data sequence and 
display codes given in Appendix I . 

If video data stream analyzer 300 does not drop any 
fields from input video data stream 301, each frame is 
15 displayed for exactly two field periods because input and 
output video data streams 301, 3 02 as well as the decoded 
sequences, all are at exactly 3 0 frames /second, or 
equivalently, 60 fields/second. However, if video data 
stream analyzer 3 00 drops some fields, the transmitted 
20 data stream from encoder 103 is no longer at 30 

frames/second. In fact, the dropping of fields can be 
random and consequently, the transmitted frame rate is a 
non-deterministic value less than 3 0 frames/second. On 
the other hand, display device 13 0 almost invariably 
25 refreshes the screen at a fixed rate of 60 fields/second. 
In such cases, therefore, decoder 121 must display some 
frames for more than the normal two field periods. 

A smart encoder can transmit the display information 
generated by video data stream analyzer 300 to a smart 
3 0 decoder which in turn can display the video sequence at 
the correct- rate and in a order that is very close to the 
original sequence . The methodology of encoding and 
transmitting, display codes is an encoder option and is not 
required to understand the principles of this invention. 
35 An example of such an encoder/decoder pair is represented 
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by the MPEG2 standard, that is currently under 
development. However, note that not all the display codes 
generated by video data stream analyzer 3 00 of this 
invention are covered by the MPEG2 standard. 
5 Since input video data stream 301 to video data 

stream analyzer 300 is at 60 fields/second and output 
video data stream 302 may be less than 60 fields/second 
synchronization between the input and output video data 
streams may be required. In one embodiment, 
10 synchronization is achieved through insertion of dead time 
periods in output video data steam 302. A dead time 
period is an interval of time, equal to a field period 
when encoder 103 does not read any data from video data 
stream analyzer 300. A special output signal in side 
15 channel 418 is used to inform encoder 103 that the data in 
the following field period has been dropped and is not to 
be read. One dead time period is required for each extra 
field period in excess of two field periods that a frame 
is to be displayed. 
20 There are other methods that can be used to achieve 

synchronization between the input and output video data 
streams. For example, the field sync preceding a dropped 
frame can be delayed instead of introducing dead time 
periods. Additionally, the rate of data output from the 
25 reordering memory 403 may or may not be changed. Several 
of these alternative are presented graphically in Figures 
6A to 6C. 

Jn Figure 6A, the side channel contains information 
telling encoder 103 not to read any information from video 

30 data stream analyzer 300 during the dead time periods 
Each of the tick marks in Figure 6A represents a field 
sync. Specifically, data is transmitted to encoder 103 
dunng field periods 601, 602 and 603. During field 
period 604, encoder 103 is instructed not to read data 

35 Encoder 103 again reads data during field periods 605 and 
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606 and then is instructed not to read data during field 
period 607. Field periods 604 and 607 correspond to 
fields that have been dropped from. input video data 
stream 301. 

5 Figure 6B is an alternative way to achieve the same 

result as that achieved by the method represented in 
Figure 6A. The field syncs preceding fields 601, 602 
and 6 03 A are generated in the normal manner and the data 
rate is the same as in Figure 6A. However, no field sync 
10 is generated at the start of field 604, which was dropped. 
The next field sync is generated at the start of 
frame 605. A field sync is generated for frame 606 and 
the next field sync is generated at the start of frame 60 8 
because field 607 was dropped. During field period 603A 
15 (Figure 6B) , which is two field periods in length, all of 
the data is transferred from video data stream 
analyzer 300 to encoder 103 during the first field period. 

In Figure 6C, the field syncs are generated in a way 
that is identical to that described for Figure 6B. 
20 However, in this embodiment, the data rate for field 

period 603B is modified so that the data in the frame 603B 
is transferred over the two field periods of field 
period 603B. The exact method of handling the mapping 
from a fixed to a variable frame rate is not required to 
5 understand the principles of this invention. The 
important aspect is simply that synchronization is 
maintained so that decoder 121 can effectively decode the 
transmitted sequence . 

Figure 7A is a process flow diagram for 
0 controller 402. In state machine transition step 701, 
state machirie 48 0 makes the transition to a new sub -state 
as indicated by a next state selector and transfers 
processing to update counter step 702 . In update counter 
step 702, the value of output counter 460 (Figure 4B) , 
5 usually referred to as output counter OC, is decremented. 
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As is explained more completely below, output counter 4 60 
is, in this embodiment, a modulo three output counter OC 
and must have a value of zero before state machine 480 
performs the steps necessary to select a field from 
5 reordering memory 4 03 for output video data stream 302. 
In scan pattern analysis step 703 (Figure 7A) , 

parameters used for the scan i-i=>i-t-,=-v-~ i 

cne scan pattern analysis are updated 

using the newly available values from statistical 
analyzer 401 for the current field. After the parameters 
10 are updated, statistical processor 490 analyzes the scan 
pattern and updates the scan pattern history in scan 
pattern register SP at each field sync . Processing 
transfers from scan pattern analysis step 703 to repeat 
frame and scene cut analysis step 704. 
IS m repeat frame and scene cut analysis step 704 

parameters and thresholds used for repeat frame and scene 
cut analyses are updated as required by controller 402 
using the newly available values from statistical 
analyzer 401 for the current field.. Scene cut 
20 register SC, repeat frame register RF and odd-even compare 
register OEC are updated at each frame sync. As explained 
more completely below, in one embodiment, the operations 
in steps 703 and 704 are intermingled to achieve more 
efficient processing. 
25 In video output sequence processing step 705, state 

machine 480 checks the value of modulo three output 
counter OC. if and only if the value of modulo three 
. output , counter OC is zero, state machine 480 executes the 
operations f.or the sub-state being processed 
30- Specifically, the addresses for the fields in the next 
frame that will be in video output data stream 302, the ■ 
display code for the next frame, the output format (i e ' ' 
field or frame) and the new value of modulo three output' 
counter OC are all determined at this time and then 
35 processing transfers to step 706. if the value of ' 
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three output counter OC is not zero, processing transfers 
directly to step 706. 

In step 706, the next sub-state is determined and the 
next state selector is updated. At the next field sync, 
5 process 700 loops from step 706 back to step 701 and the 
process is repeated for the next sub-state. 

The first two steps in process 700 are straight 
forward and do not require further explanation. However, 
steps 703, 704, and 705 are more complex and so are 

10 described in more detail. The operations in scan pattern 
detection, scene cut detection, and repeated field 
analysis are each considered in turn, and then a 
particular implementation of each process is described. 
As explained above, video data stream analyzer 3 00 

15 analyzes, in real time, input video data stream 3 01 to 

determine whether the video sequence has a progressive or 
an interlaced format. This input scan detection is done 
in a scan pattern analysis process 703 (Figure 7B) within 
controller 402. Scan pattern analysis process 703 is 

2 0 performed for each field pair in input video data 

stream 301. In index generation step 751, a plurality of 
scan pattern activity indices for a field pair are 
generated by controller 402, using parameters from 
statistical analyzer 401, i.e., the characteristics of 

25 each field pair in input video data stream 3 01 are 

monitored. In one embodiment, each scan pattern activity 
index is a strip inter-intra index. In interlaced- 
progressive analysis step 752, each of the scan pattern 
activity indices are compared to a scan pattern threshold. 

30 If a predetermined number of the scan pattern activity 
indices are greater than the scan pattern threshold, the 
field pair is identified as interlaced in step 753 and 
otherwise as progressive in step 754. 

Specifically, in one embodiment of scan pattern 

35 analysis process 703, two parameters, differential strip 
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activity DA from differential accumulator 453 and intra- 
strip activity IA from differential accumulator 454, are 
utilized. If the current field is an odd parity field, 
intra- strip activity for each strip is stored in random 
5 access memory as odd intra-strip activity IAODD-i, where 
"i" is the strip number, by controller 402 and conversely 
if the current field is an even parity field, intra-strip 
activity IA for each strip is stored in the random access 
memory as even intra-strip activity IAEVEN-i. Note that 
10 the field-pair under consideration consists of the current 
field and its immediately preceding field. Thus, one of 
the intra-strip activities represents the current field, 
while the other intra-strip activity represents the 
previous field. The differential strip activity DA for 
15 each strip represents the strips in the previous field and 
is thus the inter field difference activity measure 
between the previous field and the current field. 

An interlaced/progressive decision is made in the 
scan pattern analysis process for each pair of adjacent 
20 fields in input video data stream 301, i.e., for every 
odd-even and even-odd pair of fields. The scan pattern 
information for every odd-even and even-odd pair of fields 
is used by controller 402, as described more completely 
below. Briefly, controller 402 can re-order fields in 
25 output data stream 302 so that the odd field of one frame 
and the even field of the preceding f rame constitute a new 
frame. The even-odd pair scan pattern analysis is used to 
determine the scan pattern of the new frame. 

For each field pair in the input data stream, a 
30 plurality of inter-intra indices R-i, i. e ., scan pattern 
activity indices, are utilized to make the interlaced/ 
progressive decision. Specifically, one embodiment of 
steps 751 and 752 of scan pattern analysis process 703 is 
illustrated in Figure 7C. Initially, in step 780, a 
35 threshold counter is reset and processing transfers to 
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step 781. In step 781, differential activity DA-i for 
strip "i" is tested against a strip activity threshold 
T_m. The definition, updating, and resetting of strip 
activity threshold T_m is described more completely below. 
5 If differential activity DA-i is greater than strip 

activity threshold T-m, processing transfers to step 783 
and otherwise to step 782. 

In first set index step 782, inter-intra index R-i 
for strip "i n is set to a value less than the scan pattern 

10 threshold. Processing transfers from set index step 782 
to index test step 787. 

In sum intra-strip activity step 783, the sum of the 
odd intra-strip activity IAODD-i and even intra-strip 
activity IAEVEN-i for strip "i, " is generated. Processing 

15 transfers from sum intra-strip activity step 783 to 
step 784 . 

In sum test step 784, the sum of the odd intra-strip 
activity IAODD-i and even intra-strip activity IAEVEN-i is 
tested against a strip activity threshold T_m. If the sum 

20 of the odd intra-strip activity IAODD-i and even intra- 
strip activity IAEVEN-i is greater than strip activity 
threshold T_m, processing transfers to step 786 and 
otherwise to step 785. 

In second set index step 785 inter-intra index R-i 

25 for strip "i" is set to a value greater than the scan 
pattern threshold. Processing transfers from set index 
step 785 to index test step 787. 

In generate index 786, inter-intra index R-i is 
defined, in this embodiment, as the ratio of differential 

3 0 activity DA-i for strip »i» of the two fields to the sum 
of the odd intra-strip activity IAODD-i and even intra- 
strip activity IAEVEN-i for strip "i," i.e., 



R-i 



DA-i 

IAODD - i + I AEVEN - i 
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After generation of inter-intra index R-i for strip »i», 
processing transfers to index test step 787. 

In index test step 787, inter-intra index R-i is 
tested against the scan pattern threshold. If inter-intra 
5 index R-i is less than scan pattern threshold, processing 
transfers to last strip test step 789 and conversely to 
increment threshold counter 788, which in turn increments 
the threshold counter to indicate that inter-intra index 
R-i is. greater than the scan pattern threshold. 
10 Processing transfers from step 788 to step 789. 

Step 789 determines whether all the strips in a field 
have been processed. If additional strips remain to 
process, processing branches to step 781 and steps 781 to 
steps 788 are repeated as required. if all the strips 
15 have been processed, processing transfers from step 789 to 
step 790. 

In threshold counter check step 790, if three or more 
strips out of the fifteen strips, i.e., a predetermined 
number, have an inter-intra index R-i that exceeds unity, 
20 i.e., the scan pattern threshold, the pair of fields is 
flagged as interlaced in step 753. Otherwise, the pair is 
flagged as progressive in step 754. Specifically, the 
value of the threshold counter is checked against the 
predetermined number N and the process branches to either 
25 step 753 or step 754, as described. The embodiment of 
steps 751 and 752 given in Figure 7C is illustrative only 
and is not intended to limit the invention to the 
particular sequence of steps described. In view of this 
disclosure, those skilled in the art can implement the 
30 real, time scan pattern detection process in a variety of 
ways . 

Thus, in this embodiment, real-time scan pattern 
detection is based on generating an inter-intra index for 
each strip of two successive opposite parity fields. The 
35 index is obtained either by setting the index to a 
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particular value or by dividing the differential activity, 
metric, i.e, differential activity DA-i, between the two 
successive opposite parity fields by the sum of the 
metrics representing the intra field activities of the two 
5 fields. If three or more of the indices for strips in the 
field exceed the threshold, the frame consisting of the 
field pair is declared interlaced, else the frame is 
declared as progressive. 

As explained above, the differential activity metric 

10 measures the pixel-to-pixel difference between, adjacent 
lines of a frame because the corresponding lines of two 
successive fields form adjacent lines of that frame. The 
intra-activity metric measures the pixel-to-pixel 
difference between adjacent lines of the field. 

15 The two fields of an interlaced frame are temporally 

disjoint. Hence, the correlation between adjacent lines 
of an interlaced frame, i.e., between two lines belonging 
to different fields of the frame, is lower than the 
correlation between lines belonging to the same field, 

20 i.e., lines that were captured at approximately the same 
time. Thus, the inter-intra index is expected to be 
greater than unity for an interlaced frame. 

On the other hand, a progressive frame has all its 
lines captured at the same time instant. As a result, the 

25 correlation between adjacent lines, indicated in a way by 
the differential activity metric, is likely to be greater 
than that between alternate lines since the former are 
spatially closer. In such cases, the inter-intra index is 
likely to be less than unity. The value of three for the 

3 0 number of strips whose indices exceed unity was chosen 
empirically. This analysis is performed. oh a strip by 
strip basis because such an analysis improves the chances 
of capturing localized activity in the frame. 

In one embodiment, scan pattern register SP is an 

35 eight bit register that is used to store the scan pattern 
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history of the xast eight field-pairs. As explained above 
the scan pattern analysis process is performed after every 
field sync. Scan pattern register SP is updated as 
follows: if the field-pair is interlaced, a one is shifted 
S in from the right, i.e, entered as the least significant 
bit of register SP, else a zero is shifted in as the least 
significant bit. The most significant bit MSB of 
register SP is discarded. In this embodiment, the size of 

10 of\T n Pa " ern re9iStSr 13 SSleCted S ° that ^e number 
of bits. is egual to the number of fields that can be 

stored in reordering memory 403. 

Hence, scan pattern analysis process. 703 determines 
« real time whether a field pair in input video data 

15 TaTlT, iS - ln 3 Pr ° greSSive " interlaced format. For 
15 each field-pair the format is recorded in scan pattern 

register SP for subsequent use by state machine 480 

Repeated field and scene cut analysis processes use 

several threshold values in conjunction with strip 

20 oart^T 5 f ^ fiSld aCtivities FA - Kecall that for a 
20 particular field, field activity FA is the sum of strip 

this embodiment, the number of strips n is fifteen 

The repeated field analysis process uses both '.trip 
activities and field activity PA to check lf any ^ P 
25 successive fields of the same parity are identical, i e 

if the second field is merely a repetition of the f i rst ' ' . 
In some situations, field activity FA may be sufficient "to 
.detect repeated fields. However, in general, strip 
activities make the detection of repeated fields more 
30 robust Th e reason is that sometimes only a small portion 
of a field may change, and this change may not be captured 
by using only field activity FA, i. e , the field woull be 
flagged as a repeated field when in fact the field was not 
- repeated field. SinC e one of repeated fields is 
dropped, the erroneous flagging may affect the visual 
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quality of the display because the human eye may catch the 
change in the video sequence. Using strip activities SA 
in addition to field activity FA increases the likelihood 
of catching localized changes in the field. 
5 Two different decision criteria are used, a primary 

and a secondary criterion that are described more 
completely below, in the repeated field decision analysis. 
Preferably, both criteria are used for both odd and even 
parity fields. However, for one embodiment, in the case 

10 of odd parity fields, both of the criteria may be used in 
the repeated field decision analysis, and in the case of 
even parity fields, the primary criterion is sufficient to 
make the repeated field decision and the secondary 
criterion is not used, although the secondary criterion is 

15 evaluated. in general the repeated field decision 
analysis can be performed using only the primary 
criterion, only the secondary criterion, or the 
combinations of primary and secondary criteria just 
described. 

2 0 Prior to considering detection of repeated fields 

more completely, the detection of scene cuts is described. 
As described below, information from the scene cut 
analysis, i.e., a threshold, may be used in the repeated 
field analysis, and the repeated field analysis is used in 
25 updating thresholds in the scene cut analysis. Therefore, 
it is informative to consider the scene cut analysis and 
then return to a more detailed description of the repeated 
field analysis. 

In scene cut analysis process 850. (Figure 8A) , in 

3 0 step 851, a cumulative difference in activity between a 

first field of a given parity and a second field of the 
given parity is measured. Specifically, statistical 
analyzer 401 generates a plurality of strip activities ,- as 
described above, and controller 4 02 generates a field 
3 5 activity FA. 
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In step 852, the cumulative difference in activity, is 
used to detect a scene cut, preferably in real time. 
Specifically, a scene cut is identified if the cumulative ' 
difference is greater than a scene cut threshold 
5 Whenever there is a scene cut in input video data 

stream 301, there is a sudden jump in the value of the 
field activities FAs of the last two fields of the old 
scene. This is because, the two fields processed by 
differential accumulator 452 are now drastically 
10 different. Thus, by comparing the field activity of each 
field to typical field activities for the scene, i.e, a 
threshold, a change in scene can be detected. 

In this embodiment of process 850, typical field 
activities are maintained for odd parity fields and even 
15 parity fields. Specifically, an odd scene cut 

threshold T_sl and an even scene cut threshold T_s2 are 
used to determine when a scene cut occurred. As explained 
more completely below, odd scene cut threshold T_sl is a 
runnmg average of the field activities for the odd parity 
20 fields in a scene. Similarly, even scene cut threshold 
T_s2 is a running average of the field activities for the 
even parity fields in the scene . 

If the current field is odd, field activity FA is 
compared to odd scene cut threshold T_si i n step 852 
25 Specifically, to assure that a scene cut has occurred and . 
the change in field activity FA is not caused by noise or 
some other spurious activity, for an odd parity field, the 
f lel f aCtivifc y FA must ^ substantially greater than odd 
■scene cut threshold T_sl. m one embodiment, field 
30 activity FA of an odd parity field must be greater than a 
constant K_sl times scene cut threshold T_sl, i.e., 
K_sl* T _sl, and greater than a constant N_sl times 'field 
activity threshold T_n ( which is described more completely 
below, i.e., greater than N_sl*T_n. Xn one embodiment 
35 constant K_sl is 1.75 and constant N_sl is 3 5 The 
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requirement that field activity FA is greater than 
N_sl*T_n assures that field activity FA has a minimum 
value before a scene cut is declared. Hence, when field 
activity FA of an odd parity field is greater than 
5 K_sl*T_sl and greater than N_sl*T_n, the current odd 
parity field and the next odd parity field are declared, 
i.e., identified, as belonging to different scenes. Hence, 
in this embodiment, the threshold has two components, and 
field activity FA must be greater than both components . 

10 Similarly, for an even parity field, the field 

activity FA must be greater than even scene cut 
threshold T_s2 . Preferably, field activity FA must be 
greater than a constant K_s2 times even scene cut 
thresholds T_s2 , i.e., K_s2*T_s2, and greater than a 

15 constant N_s2 times field activity threshold T_n, i.e, 
N_s2*T__n. In one embodiment, constant K_s2 is 1.75 and 
constant N_s2 is 3.5. Hence, when field activity FA for 
an even parity field is greater than K_s2*T_s2 and greater 
than N_s2*T_n, the current even parity field and the next 

20 even parity field are declared, i.e., identified, as 

belonging to different scenes. Hence, in this embodiment, 
the threshold has two components, and field activity FA 
must be greater than both components . 

In one embodiment, scene cut register SC is an eight 

25 bit register that is used to store the scene cut history 
of the last eight fields. At each frame boundary, scene 
cut register SC is updated by shifting in two bits from 
the right as explained below. The two most significant 
bits of register SC are discarded. 

3 0 Scene cut register SC is updated at each frame 

boundary as follows : if a scene cut was detected at the 
last odd parity field, the first bit shifted in is a one, 
and conversely a zero is shifted in; and if a scene cut 
was detected at the last even parity field, the second bit 

3 5 shifted in is a one and conversely a zero is shifted in. 
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Thus, analysis of -the bit pattern, in scan cut register SC 
identifies whether a scene cut occurred within a frame or 
at a frame boundary. 

Scene cut thresholds T_sl and T_s2, are running 
5 parameters and at each frame boundary, an analysis is 
performed to determine how to update the thresholds 
Process 800 illustrated in Figure 8B is used to analyze 
odd scene cut threshold T_sl. m process 800, various 
criteria and flags are checked. As explained more 
10 completely below, each piece of information used in 
process 800 is available at the time process 800 is 
performed. 

In scene cut at last odd parity field step 8 02, an 
odd parity field scene cut flag is checked to determine if 
15 there was a scene cut at the previous odd parity field 
Each time a scene cut is detected at an odd parity field 
in process 850 (Figure 8A) , the odd parity field scene cut 
flag is set and conversely. Alternatively, the second bit 
xn scene cut register SC could be examined. if the odd 
20 parity field scene cut flag is set, processing transfers 
to step 804 and conversely to step 812. 

If a scene cut was detected at the last odd parity 
fxeld, step 804 determines whether the primary criterion 
for repeated field detection for the current odd parity 
25 field is satisfied. if the primary criterion is not 

satisfied, processing transfers to a first update 
• step 808. update step 808 sets odd scene cut 
| threshold T_si equal to field activity FA for the current 
odd parity field. 

30 Conversely if the primary criterion for the current 

odd parity field is satisfied, processing transfers from 
step 804 to step 807. m step 807, if the primary 
criterion for the next even parity field was satisfied 
processing transfers from step 807 to step 808 that was 

35 3 ust described. Here, the next even parity field means 
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the field immediately following the current odd parity- 
field. In step 807, if the primary criterion for the next 
even parity field was not satisfied, processing transfers 
from step 807 to step 809. Step 809 sets odd scene cut 
5 threshold T_sl to the default value. In one embodiment, 
the default value for odd scene cut threshold T si is 
40.667*N_f, where N_f is the number of pixels in a field. 
Both step 809 and step 808 transfer to end step 810 upon 
completion. The check performed by step 807 distinguishes 
10 between the case where both the odd and even parity fields 
are repeated and the case where only the odd parity field 
is repeated. 

If a scene cut was not detected at the last odd 
parity field in step 802, step 812 checks to determine if 

15 the primary criterion for repeated field detection for the 
current odd parity field is satisfied. If the primary 
criterion is satisfied, processing transfers to step 813 
and otherwise to step 816. 

Primary criterion satisfied for next even parity 

20 field step 813 checks to determine whether the primary 
criterion for the next even parity field was satisfied. 
If the primary criterion was not satisfied processing 
transfers from step 813 to end step 814 and so even scene 
cut threshold T_sl is not changed. If the primary 

25 criterion was satisfied, processing transfers from, 
step 813 to scene cut at current odd parity field 
step 816 . 

Step 816 compares odd scene cut threshold T_sl with 
field activity FA for the current odd parity field, as 
30 described above. If a scene cut is detected, processing 
transfers frbm step 816 to end step 822 so that in this 
case, threshold T_sl is not changed. 

If step 816 shows that there is no scene cut 
processing transfers to step 818. Step 818 determines 
35 whether odd scene cut threshold T_sl is equal to the 
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default value. if odd scene cut threshold T_ sl i s equal 
to the default value, processing transfers from step 818 
to a second update step 824 which in turn sets odd scene 
cut threshold T_sl equal to field activity FA for the 
5 current field. 

If odd scene cut threshold T_sl is not equal to the 
default value, processing transfers from step 818 to a 
filter step 820. In step 820, a recursive filter is used 
to determine the new value of odd scene cut 
10 threshold T_si. m one embodiment, the new odd scene cut 
threshold T_sl is: 

T_sl = (or*(T_sl) + (l-a)*FA). 

constant a i s in the range from zerQ tQ Qne ^ ^ ^ 
embodiment is 0.!. Upon completion of step 820, processing 
15 transfers to end step 822. m end steps 8X0, 814 and 822, 
if odd scene cut threshold T_si is less than 2 * n f 
threshold T_sl is set to 2 * N_f, i.e., the threshold is 
clamped at a minimum level. 

The updating of even scene cut threshold T s2 is 
20 similar to that of odd scene cut threshold T_sl~ 

Process 860 illustrated in Figure 8C is used to analyze 
even scene cut threshold T_s2. Again, as explained more 
completely below, all the information needed in 
process 860 is available at the time the process is 
25 performed. 

In scene cut at last even parity field step 862, an 
even parity. field scene cut flag is checked to determine 
if there was a scene cut at the previous even parity 
field. Each time a scene cut is detected at an even 
30 parity field in process 850 (Figure 8A) , the even parity 
field scene cut flag is set and conversely. 
Alternatively, the first bit in scene cut register SC 
could be examined. if the even parity field scene cut 
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flag is set, processing transfers from step 862 to 
step 864 and conversely to step 872. 

If a scene cut was detected at the last even parity 
field, step 8 64 determines whether the primary criterion 
5 for repeated field detection for the current even parity 
field is satisfied. If the primary criterion is not 
satisfied, processing transfers to a first update 
step 868. Update step 868 sets even scene cut 
threshold T_s2 equal to field activity FA for the current 
10 even parity field. 

Conversely, if the primary criterion for the current 
even parity field is satisfied, processing transfers from 
step 864 to step 867. In step 867, if the primary 
criterion for the previous odd parity field was satisfied, 
15 processing transfers from step 8 67 to step 8 68 that was 
just described. Conversely, in step 8 67, if the primary 
criterion for the previous odd parity field was not 
satisfied, processing transfers from step 867 to step 866. 
Step 8 66 sets even scene cut threshold T_s2 to the default 
20 value. Both step 866 and step 868 transfer to end 

step 8 70 upon completion. The check performed by step 867 
distinguishes between the case where both the odd and even 
parity fields are repeated and the case where only the 
even parity field is repeated. 
25 if a scene cut was not detected at the last even 

parity field in step 862, step 872 checks to determine if 
the primary criterion for repeated field detection for the 
current even parity field is satisfied. If the primary 
criterion is satisfied, processing transfers to step 873 
3 0 and otherwise to step 876. 

■Primary criterion satisfied for previous odd parity 
field step 873 checks to determine whether the primary 
criterion for the previous odd parity field was satisfied. 
If the primary criterion was not satisfied, processing 
35 transfers from step 873 to end step 874 and so even scene 
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cut threshold T_s2 is not changed. if the priraary 
crxterxon was satisfied, processing transfers from 
step 873 to scene cut at current. even parity field 
step 876. 

5 Step 876 compares even scene cut threshold T s2 with 

de'cribel^;^ '* ^ ^ fi * Id ' •* 

tr^fers I " * ^ ^ ±S dete <^ . Pressing 

transfers from step 876 to end step 882 so that in this 
case, threshold T_s2 is not changed. 
10 If st «P 876 shows that there is no scene cut 

whe°the S r ing tr&nSferS ^ ^ ^ ^ 878 

whether even scene cut threshold T_s2 is equal to the 

default value. m one embodiment, the default value for 

15 t7 n t h SCen \ CU * threSh ° ld T " S2 " 40 " 667 * N - f ' N f 

thr , ^ ° f PiXSlS ^ 3 field " " — — e cut- 

threshold T_ s2 is equal to the default value, processing 

step 878 to a second update ^ 

in turn sets even scene cut threshold T s2 equal to field 
activity FA for the current field. . " 

20 m n " SVen SC6ne ° Ut thresh °ld T_s2 is not equal to the 
default value/ processing transfers from £ 

filter step 880. In step 880, a recursive filter is used 
to determine the new value .of even scene cut 

2 S tt~ ::; 2 2 - is : n ~ ~ - - even scene cut 

T_s2 = (0f*(T_s2) + (l-cO*FA). 

eZd ant * " " rSn9e fr ° m Zer ° t0 ° ne - d i» one 

embedment xs 0.1. Dpo n completion of step 880 

874 and 882, xf even scene cut threshold T s2 is less 
than 2 * N_f , threshold T_s2 is set to 2 * N f , i e the 
threshold is clamped at a minimum level 



£>«; i »<■■«!> a t-» 



WO 95/15659 



PCT/US94/13287 



- 59 - 

Hence, under "normal" conditions, thresholds T_sl 
and T__s2 are updated using a "recursive filter" . The 
numbers "0.9" and "0.1" currently used in the recursive 
filters were obtained empirically. However, "abnormal" 
5 conditions, i.e., sudden deviations in field activity FA, 
are ignored. Such sudden deviations can be introduced by 
repeated fields when field activity FA is very small and 
hence not a realistic indicator of the field- to-field 
activity, and by scene cuts when field activity FA is 
10 suddenly very high, and again does not indicate the normal 
field-to-field activity. Hence, the numerous tests in 
processes 800 and 860 are utilized to detect possibly 
abnormal conditions and prevent such conditions from 
biasing the thresholds used in the scene cut detection 
15 process of this invention. 

Returning to the repeated field detection and the 
evaluation of the primary and secondary criteria, two 
different thresholds T_n and Tjn, are used in evaluating 
whether the primary criterion is satisfied. Each strip 
20 activity SA-i for the pair of fields being processed is 
compared to strip activity threshold T_m, and field 
activity FA for the pair of fields being processed is 
compared to field activity threshold T_n. Recall that the 
pair of fields being processed is the current field and 
25 the next field having the same parity as the current 

field. The strip activities and the field activity are 
associated with the current field. If either two or more 
strip activities exceed strip activity threshold T__m, or 
field activity FA exceeds field activity threshold T_n, . 
30 the primary criterion is not satisfied; otherwise, the 

primary criterion is satisfied. Specifically, the primary 
criterion is satisfied if field activity FA is less than 
or equal to field activity threshold T_n, and at least 
fourteen strip activities are less than or equal to strip 
35 activity threshold T_m. 
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If the primary criterion is not satisfied, the two 
fields under consideration, whether of odd or even parity 
are different. If the primary criterion is satisfied for' 
an even parity field and the primary criterion is not 
5 satisfied for the odd parity field of the same frame, i e 
the previous field, the even parity field is a repeated 
field. if the primary criterion is satisfied for both the 
odd and even parity fields of a frame, a secondary 
criterion is evaluated for the even parity field If the 
10 secondary criterion is true, i.e, satisfied, the even 
panty field is flagged as a repeated field. In another 
embodiment, if the primary criterion is . satisfied for the 
even parity field, the field is flagged as repeated. 

In the case of odd parity fields, if the primary 
15 criterion is found to be true for an odd parity field, and 
the primary criterion for the following even parity field 
as false, the odd parity field is flagged, i.e 
identified, as a repeated field. If the primary criterion 
for the following even parity field is also found to be 
2 0 true, the secondary criterion is checked to arrive at a 
final decision. If the secondary criterion is satisfied 
the odd parity field is flagged as a repeated field. 

The secondary criterion for odd and even parity 
fields is examined after odd scene cut threshold T si and 
25. even scene cut threshold T_s2 are updated using processes 
800 and 860, respectively, as described above . However 
the secondary criterion is tested only if the appropriate 
: -scene cut threshold, as explained below, is not equal to 
the default value. If the appropriate scene cut threshold 
30 equals the default value, the secondary criterion is 
declared not satisfied. Otherwise, the secondary 
criterion compares a measure of field activity for the 
field under consideration, e.g., a scaled version of field 
activity FA, with the scene-cut threshold corresponding to 
35 the other parity fields. 
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Specifically, in one embodiment, if the field under 
consideration is of odd parity, the secondary criterion 
compares a constant K times field activity FA with even 
scene cut threshold T_s2, which, is a running even parity 
5 field activity parameter. Thus, a measure of the field 
activity is compared with the threshold. In one 
embodiment, the value of constant K was taken as 1.75. 
Conversely, if the field under consideration is of even 
parity, the. secondary criterion compares constant K times 

10 field activity FA with odd scene cut threshold T_sl, which 
is a running odd parity field activity parameter. Again, 
a measure of the field activity is compared with the 
threshold. If the product of constant K and field 
activity FA is less than the threshold value the secondary 

15 criterion is satisfied for either odd or even parity 

fields. However, if the product of K and field activity 
FA is less than the appropriate threshold value for both 
the even and odd parity fields, the secondary criterion 
for both the even and odd parity fields is declared 

20 unsatisfied, i.e., not true. In other, words, if a frame 
is repeated, the secondary criterion for the two fields in 
the frame are declared unsatisfied by controller 402. 

Thus, for an odd parity field, if the primary 
criterion for odd parity field and the even parity field 

25 is satisfied, and the secondary criterion for the odd 

parity field is satisfied,' the odd parity field is flagged 
as a repeated field. Similarly, for an even parity, field, 
if the primary criterion for odd parity field and the even 
parity field of a frame is satisfied, and the secondary 

3 0 criterion for the even parity field is satisfied, the even 
parity field is flagged as a repeated field. 

The repeated field detection process, as described 
above, is illustrated in Figure 9A as repeated field 
detection process 900. PC true step 980 checks to 

35 determine whether field activity FA is less than or equal 
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to field activity threshold T_n, and at least fourteen 
strip activities are less than or equal to strip activity 
threshold T_m. if either test is not satisfied, 
processing transfers to fields different step 981 where 
5 the current field is identified as an unrepeated field, 
i.e., the primary criterion field flag is not set If 
both tests are satisfied, a flag is set indicating that 
the primary criterion for the field is true and processing 
transfers from step 980 to step 982. 
10 m PC true for other field test step 982, a check is 

made to determine whether the other field in the same 
frame as the current field has the primary criterion flag 
set. Note that for the odd parity field in the frame 
th 1S test is performed after the even parity field has 
15 entered reordering memory 403. if the primary criterion 
flag for the other field in the frame is not set 
processing transfers from step 982 to step 983 where the 
field ls identified, e.g., flagged, as a repeated field 
If the primary criterion flag for the other field in the 
-0 frame ls set, processing transfers from step 982 to 
step 984. 

In scene cut threshold test step 984, the scene cut 
threshold for the parity opposite to the parity of the 
current field is checked to determine whether that scene 

5 cut threshold has the default value. if the scene cut 
threshold has the default value processing transfers from 
step 984 to fields different step 985 which in turn does 
not flag the current field as a repeated field. If the 
scene cut . threshold does not have the default value 

3 processing transfers from step 984 to. step 986. 

^ SC trUS StSP 986 ' ^ secondarv criterion is 

checked, as described above, to determine whether the 
secondary criterion is true. If the secondary criterion : 
is not satisfied, processing transfers from step 986 to 
step 987, which in turn identifies the current field as an 
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unrepeated field. If the secondary criterion is 
satisfied, a secondary criterion flag is set and 
processing transfers from step 986 to step 988. 

In SC true for- frame test step 988, the secondary 
5 criterion flag for both fields in the frame is tested to 
determine whether both flags are set. If one secondary 
criterion flag is not set, processing transfers to 
step 989, which in turn sets the repeated field flag for 
the current field. If both secondary criterion flags are 
10 set for the frame, processing transfers from step 988 to 
step 990. 

In step 990, the secondary criterion flags for both 
fields in the frame are set to false. In this case, 
either the frame is a repeated frame or neither field in 

15 the frame is repeated. State machine 480 makes the 

determination by checking whether the primary criterion 
for both fields of the frame following the next frame are 
satisfied and the secondary criterion are false. 

Prior to considering another embodiment of the 

20 repeated field detection process in more detail, it is 
informative to consider the actual sequence in which the 
scan pattern, scene cut, and repeated field analyses are 
performed since now all the terms and thresholds have been 
defined. After every field sync, statistical 

25 processor 490 determines all of the parameters pertaining 
to the field that most recently entered reordering 
memory 4 03. 

In this embodiment of steps 703 to 705, in primary 
criterion analysis step 760, (Figure 7D> the primary 

3 0 criterion for the field is checked and the result is 

stored in a temporary variable, e.g., a primary criterion- 
flag is set if the primary criterion is satisfied. Next, 
scan pattern analysis process 703 (Figures 7B and 7C). is 
performed. Primary criterion satisfied check. 761 

3 5 determines whether the temporary variable was set in 
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step 760. If the temporary variable was set, processing 
transfers from step 761 to step 762, which in turn updates 
strip activity threshold T_ m and field activity threshold 
T_n, as explained more completely below. 
5 Step 762 transfers processing to step 763 as does 

step 761 if the temporary variable was not set. In 
step 7 62( scan pattern register SP is updated. of course 
this step could be combined with step 703 . 

Even P ari *y field check 764 determines whether both 
10 the odd and even parity fields of a frame have entered 
reordering memory 403. if both fields have not entered 
reordering memory 403, step 764 transfers to step 760 and 
steps 760, 703, 761, 762 and 763 are repeated for the even 
parxty field. when both fields have entered reordering 
15 memory 703, step 764 transfers to perform scene cut 
analysis step 770 . 

Step 770 performs scene cut detection process 850 
(Figure 8A) , as described above, and transfers processing 
to update scene cut thresholds step 771. step 771 
20 performs processes 800 and 860, as described above to 
update scene cut thresholds T_sl and T_s2 . Next in 
step 772, the secondary criterion is evaluated for the odd 
and even parity fields, e.g. steps 982 to 990 of 
process 900 are completed. Upon completion of step 770 
25 processing transfers to update registers step 775. 

In step 775, since a frame boundary has occurred 
registers 470 with the exception of scan pattern 

re ^ster SP are updated as described above. Finally as 
described more, completely below, if the output counter has 
30 a zero value, step 776 transfers to step 70S where the 
appropriate operations are performed to send information 
to encoder 103 and otherwise to step 706. 

The use of another embodiment of the repeated field 
detection process is illustrated in Figures 9B and 9C for 
a cartoon input video sequence and a telecine input video 



f 



WO 95/15659 



PCT/US94/13287 



- 65 - 

sequence. In this embodiment, only the primary criterion 
is used to determine whether an even parity field is 
repeated while both the primary and secondary criteria are 
used to determine whether an odd parity field is repeated. 
5 For purposes of illustration only five frames 901 

to 905 are considered. Each frame 901 to 905 contains two 
fields. For example, frame 901 contains an odd parity 
field 901A and an even parity field 901B. Sometimes in • 
this discussion the reference numerals for the fields are 
10 not used, but rather the content of the field is used. 
Specifically, as shown in Figures 9B and 9C, the cartoon 
video sequence is : 

ala2 ala2 blb2 blb2 clc2; 

and the telecine video sequence is : 

15 ala2 alb2 blc2 clc2 dld2 . 

The processing of the cartoon video sequence is 
considered first and then the telecine video sequence is 
considered. As described above, as each strip of fields- 
al, a2, and a3 moves through field memories. 4 50 and 451, 

20 differential accumulator 452 is producing a strip 

activity SA-i for that strip. When a strip is processed 
and loaded into reordering memory 403 , an interrupt is 
sent to controller 402, and statistical processor 490 
reads the strip activity SA-i for the field. For fields . 

25 901A and 902A, which are both al, the strip activities 
SA13 are stored in box 913, which, in one embodiment, is 
memory accessed by controller 402. When all the strips of 
field al are loaded in reordering memory 4 03, field 
activity FA13 is also generated by statistical 

30 processor 490 and repeated field detection process 950 is 
initiated. Note that in Figure 9B process 950 shows the 
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secondary criterion check immediately following the 
primary criterion check. However, in one embodiment 
(Fig. 7D) the two checks are not contiguous. 

Since fields 901A and 902A are the same, each strip 
5 activity in strip activities SA13 is approximately zero 
and so field activity FA13 is also approximately zero. 
Thus, in strip activity test 952A, which compares each of 
strip activities SA13 to strip activity threshold T_m, 
none of the strip activities exceeded the threshold. 
10 Similarly, in field activity test 954A, which compares 

field activity FA13 to field activity threshold T_n, field 
activity FA13 is less than threshold T_n. 

PC true step 956A checks to determine whether two or 
more strip activities exceeded threshold T_m in step 952A 
15 or field activity FA13 exceeded threshold T_n. Since 
neither threshold was exceeded, the primary criterion is 
satisfied and processing transfers from PC true step 956A 
to even parity check step 960A. Since field 901A and 902A 
are odd parity, even parity check step 960A transfers to 
20 PC even parity field true check step 964A. Processing in 
step 964A requires information from the next even parity 
field and so step 964A pauses until that information is 
available . 

For fields 901B and 902B, which are both a2, the 
25 strip activities SA24 are stored in box 924, which, in one 
embodxment, is memory accessed by controller 402. When 
all the strips of field a2 are entered in reordering 
memory 403, field activity FA24 is generated by 
statistical processor 490 and repeated field detection 
3 0 process 950B is initiated. Repeated field detection 

processes 950A and 950B are the same process. However, to 
facilitate discussion of the process, different reference 
numerals are used for the analysis of odd and even parity 
field pairs. 
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Since fields 901B and 902B are the same, each strip 
activity in strip activities SA24 is approximately zero 
and so field activity FA24 is also approximately zero. 
Thus, in strip activity test 952B, which compares each of 
5 strip activities SA24 to strip activity threshold T_m, 
none of the strip activities exceed the threshold. 
Similarly, in field activity test 954B, which compares 
field activity FA24 to field activity threshold T_n, field 
activity FA24 is less than threshold T_n. 

10 PC true step 956B checks to determine whether two or 

more strip activities exceed threshold T_m in step 952B or 
field activity FA24 exceeded threshold T_n. Since neither 
threshold was exceeded, the primary criterion is satisfied 
and processing transfers from PC true step 95 6B to even 

15 parity check step 960B. Since fields 901B and 902B are 
even parity, even parity check step 960B transfers to 
repeat step 962B which in turn marks, i.e., identifies, 
field 901B as a repeated field. The result of step 956B 
is also supplied to step 964A. Note that as indicated 

20 above, in this embodiment for even parity fields only the 
primary criterion is needed to determine whether the field 
is a repeated field. Nevertheless the secondary criterion 
is evaluated. In Figure 9B, the evaluation of the 
secondary criterion for even parity fields is not 

25 illustrated. 

In step 964A, when the primary criterion for the even 
parity field is true, as in step 960B, processing 
transfers to secondary criterion true step 970A. In 
step 970A, field activity FA13 is multiplied by constant K 

3 0 and the product compared to even scene cut threshold T_s2 . 
Since field activity FA13 is approximately zero, 
multiplication of zero by constant K gives a product that 
is still approximately zero. Consequently, the product is 
less than even scene cut threshold T_s2 . Since the 

35 secondary criterion was true for both fields in the frame, 
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controller 402 sets the secondary criterion for both 
fields in the frame to false. Processing transfers from 
step 970A to step 974A. 

In step 974A, since the primary criterion for odd 
5 parity field 901A was found to be satisfied, the first bit 
from the right shifted into repeated field register RF is 
a one. Similarly, since the primary criterion for even 
parity field 901B was satisfied, the second bit shifted 
into repeated field register RF from the right is also 
10 one. since the secondary criterion was not satisfied for 
both fields 901A and 901B, the first bit shifted into odd 
even compare register OEC is a zero, as is the second bit 

The default values of field activity threshold T n 
and strip activity threshold T_m, in this embodiment Is 
15 well as for process 900 (Figure 9A) , are 4*N_f and 6*N s 
respectively, where N_s and N_f are the number of pixels' 
m a strip and a field, respectively. These default 
values are assigned during power-up or whenever a scene 
cut is detected. The values of strip activity threshold 
20 T_m and field activity threshold T_n are adjusted every 
t lme the primary criterion is satisfied as indicated 
above in Figure 7D. Whenever, the primary criterion is 
satisfied for a field, field activity FA for that field is 
compared with the value of a stored maximum value of the 
25 field activity up to this time. Initially and at scene 
cuts, the stored maximum value of the field activity is 
initialized to zero. Thus, field activity FA13 is 
compared to the stored maximum value of zero. Assuming 
that field activity FA13 is greater than zero, the value 
30 of fxeld activity FA13 is stored as the maximum field 
activity. 

Next field activity FA24 is compared to the stored 
maximum .field activity, i.e., field activity FA13, and the 
larger of the two field activities is stored as the 
35 maximum field activity. After each change in the stored 
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maximum field activity, field activity threshold T_n is 
set equal to 1.15 times the stored maximum field activity, 
i.e., field activity threshold T_n is expressed in terms 
of the stored maximum field activity. Strip activity 
5 threshold T_m is set equal to 1.2 times field activity 
threshold T_n times the number of pixels in a strip N_s 
divided by the number of pixels in a field N_f, i.e., the 
new strip activity threshold is expressed in terms of the 
field activity threshold. The new values of field 

10 activity threshold T_n and strip activity threshold T_m 
are required to be in the range [3.5*N_f, 4.2*N_f] and 
[5.5*N__s, 6.25*N_s], respectively. Thus, if the new 
threshold is outside the specified range, the threshold is 
set to the appropriate endpoint of the range . In this 

15 embodiment, the constants 1.2, 1-15 and the ranges for the 
thresholds have been empirically determined. Moreover, 
the particular values used in the strip and field activity 
thresholds are illustrative only and are not intended to 
limit the invention to the particular values given. An 

2 0 important aspect is that the thresholds are defined so 

that as the frame size changes the thresholds are adjusted 
accordingly. Hence, the processes of this invention are 
not dependent upon a particular frame size. 

After frame 901 is processed, frame 902 is processed. 
25 In Figure 9B, for clarity, repeated field detection 

processes 950A, 950B are shown connected only to boxes 913 
and 924. However, as will be understood by those skilled 
in the art, field detection processes 950A, 950B are 
repetitively used to process the data in subsequent boxes 

3 0 924, 935, . . . Herein, process 950A is used for odd 

parity fields and process 950B for even parity fields. 

For fields 902 A and 903A; which are al and bl 
respectively, the strip activities SA3 5 and field 
activity FA35 are stored in box 935. Since fields 902A 
35 and 903A are different, at least two of the strip 
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activities in strip activities SA3S are large and 
consequently field activity FA35 is also large. Thus in 
strip activity test 952A, which compares each of strip 
activities SA35 to strip activity threshold T „, at least 
5 two of the strip activities exceed the threshold 

Similarly, in field activity test 954A, which compares 
field activity FA35 to field activity threshold T n, field 
activity FA35 is greater than threshold T_n. 

PC true step 956A checks to determine whether two or 
10 more strip activities exceed threshold T_m in step 952A or 
field activity FA13 exceeded threshold T_n. since both 
thresholds were exceeded, the primary criterion is not 
satisfied and processing transfers from PC true step 956A 
to dxfferent step 958A. 

15 In different ste P 9 58A, field 902A is fl agged as a 

different field. This effectively terminates 
process 950A. Nevertheless, as indicated above the 
secondary criterion is evaluated and found not to be 
satisfied. 

20 For fields 902B and 903B. which are a2 and b2 

respectively, strip activities SA46 are stored in box 946 
Since f le i ds 902B and 903B are different> ^ • 

criterion is not satisfied and so step 956B marks 
field 902B as a different field. 

902A Hen ^; D frame been P rocess ^ -d neither field 

902A or 902B is a repeated field, -since the primary 

criterion for odd parity field 902A was not satisfied, the 

first bit shifted into repeated field register rf is a 

zero. Similarly, since the primary criterion for even 

30 parity field 902B was not satisfied, the second bit 

shifted into repeated field register RF is also a zero 

Since the secondary criterion was satisfied for neither of 

fields 901A and 901B, the first bit shifted into odd evef 

compare register OEC is a 2 ero as is the second bit 
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Since the primary criterion was not satisfied for either 
field, thresholds T_n and T_m are not updated. 

The processing of the remainder of the fields in the 
cartoon sequence are equivalent to that just described and 
5 so will be apparent to those skilled in the art in view of 
the above discussion. The pattern of bits in repeated 
field register RF with a pair of bits set followed by a 
pair of bits that are not set is indicative of the cartoon 
sequence . 

10 The repeated field processing of the telecine video 

sequence is similar to the cartoon sequence, when the 
differences in the two sequences are considered. 
Nevertheless, the repeated field processing of the 
telecine sequence is considered in detail. 

15 Since fields 901A and 902A in the telecine video 

sequence are the same, each strip activity in strip 
activities SA13 is approximately zero and so field 
activity FA13 is also approximately zero. Thus, in strip 
activity test 952A, which compares each of the strip 

20 activities in SA13 to strip activity threshold T_m, none 
of the strip activities exceed the threshold. Similarly, 
in field activity test 954A, which compares field 
activity FA13 to field activity threshold T_n, field 
activity FA13 is less than threshold T_n. 

25 PC true step 956A checks to determine whether two or 

more strip activities exceed threshold T_m in step 9 52 A or. 
field activity FA13 exceeded threshold T_n. Since neither 
threshold was exceeded, the primary criterion is satisfied 
and processing transfers from PC true step 95 6 A to even 

30 parity check step 960A. Since field 901A and 902A are odd 
parity, even parity check step 960A transfers to PC even 
parity field true check step 964A. Processing in . 
step 964A requires information from the next pair of 
frames and so step 964A pauses until that information is 

35 available. 
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For fields 901B and 902B, which are a2 and b2 
respectively, the strip activities SA24 are stored in 
box 924 which, in one embodiment, is memory accessed by 
controller 402. when all the strips of field a2 have 
entered reordering memory 403, field activity PA24 is also 
deterged and repeated field detection process 950B is 
initiated. Repeated field detection processes 950A 
and 950B are the same process. However to facilitate 

10 at SCUSS H°? °' *** Pr ° CeSS ' di « erent -ference numeral 
10 are used for the analysis of odd and even parity field 

pairs . 

Since fields 901B and 902B are different, several 
strxp activities in strip activities SA24 are most likely 
large and so field activity FA24 is also large. Thus Z 
St ~* ~^ity test 9S2B, which compares each of tnTst'", 
activities to strip activity threshold T_m, at least Z I 

1 STirr?.^ t±V±t± " eXCeed ^ Similarly, 
m field activity test 954B, which compares field 

20 ^ 1Vi . ty FA24 ^ fleld aCtiVlty T n, field 

20 activity FA2 4 is considered greater than threshold T n 

more 11 V™ ' "?» 956B ° hecks to **ermi n e whether two'or 

Zl * tL * S eXCeed Str±P aCtiVit ^ threshold T m 

^ step 952B or field activity FA24 exceeded field 

25 "lT ty threSh ° ld T " n - both were exceeded, the 

25 primary criterion is not satisfied and processing 

transfers from PC true step 956B to. different step 958B 
whxch f l ags fields 90XB and 902B as different and this 
effectively terminates repeated field detection 
\ process 950B. Note that as indicated above in this 
30 embodiment for even parity fields, only th^rLa^ 
, criterion is needed to determine whether a field is a 

e^rd""" N — ' * ™y criterion is 
The result of step 956B is also supplied to 
35 step 964A. In step 96 4A, when the primary criterion for 
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the even parity field is not true as in step 956B, 
processing transfers to repeat step 966A. In step 966A, 
field 901A is marked as a repeated field. This 
effectively terminates process 950A. Nevertheless, the 
5 secondary criteria is evaluated and found to be true. 

Hence, frame 901 has been processed and field 90 1A is 
a repeated field, but field 901B in not a repeated field. 
Since the primary criterion for odd parity field 901A was 
found to be satisfied, the first bit shifted into repeated 

10 field register RF is a one. Since the primary criterion 
for even parity field 901B was not satisfied, the second 
bit shifted into repeated field register RF is a zero. 
Since the secondary criterion was satisfied for field 901A 
but not field 9 01B, the first bit shifted into odd even 

15 compare register OEC is a one and the second bit is a 
zero. The values of strip activity threshold T_m and 
field activity threshold T_n are updated as described 
above, because the primary criterion for field 901A was 
satisfied. 

20 After frame 901 is processed, frame 902 is processed. 

For fields 902A and 903A, which are al and bl 
respectively, the strip activities SA3 5 and field 
activity FA35 are stored in box 935. Since fields 902A 
and 903A are different, at least two of the strip 

25 activities in strip activities SA35 are large and 

consequently field activity FA35 is also large. Thus,, in 
strip activity test 952A, which compares each of strip 
activities SA35 to strip activity threshold T_m, at least, 
two of the strip activities exceed the threshold. . 

30 Similarly, in field activity test 954A, which compares 

field activity FA35 to field activity threshold T_n, field 
activity FA3 5 is greater than threshold T_n. 

PC true step 956A checks to determine whether two or 
more strip activities exceed threshold T_m in step 952A or 

35 field activity FA3 5 exceeded threshold T n. Since both 
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thresholds were exceeded, the primary criterion is not 
satisfied and processing transfers from PC true step 95GA 
to different step 958A. 

in different step 958A, field 902A is fi agged as a 
5 different field. This effectively terminates 

process 950A. Nevertheless, as indicated above, the 
secondary criterion is evaluated and found not to be 
satisfied. 

For fields 902B and 903B, which are b2 and c2 
10 respectively, the strip activities SA46 are stored in 
box 946. since fields 902B and 903B are different, the 

lllZlnT teri ° n ±S Satisfied and «° ^p .9588 flags 

field 902B as a different field. 

Hence, frame 902 has been processed and neither 
X5 field 902A or 902B is a repeated field. S i„ ce the primary 
criterion for odd parity field 902A was not satisfied, the 
first b lt shifted into repeated field register RF is a 
zero. Similarly, since the primary criterion for even 
parity field 902B was not satisfied, the second bit 
20 shafted into repeated field register RF is also a zero 

Since the secondary criterion was satisfied for neither of 
field, 902A and 902B, the first bit shifted into odd evel 
compare register DEC is a zero as is the second bit 
Since the primary criterion was not satisfied for either 
25 field, thresholds T_n and Tm are not updated 

For ^T/ frame " Pr ° CeSSed < *«- 503 is processed. 

For fields 903A and 904A, which are bl and cl 

respectively, the strip activities SA57 and field 

30 r\ ± r ty FA5? St ° red iD b ° X 957 »« • Since 

30 fields 903A and 904A are different, at least two of the 

strip activities in strip activities SA57 are large and 

consequently field activity FA57 is also large. Thus in 

strip activity test 952A, which compares each of strip 

activities SA57 to strip activity threshold T_m, at least 

3 5 two of the strip activities exceed the threshold 
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Similarly, in field activity test 954A, which compares 
field activity FA57 to field activity threshold T_n, field 
activity FA57 is greater than threshold T_n. 

PC true step 956A checks to determine whether two or 
5 more strip activities exceed threshold T_m in step 952A or 
field activity FA57 exceeded threshold T_n. Since both 
thresholds were exceeded, the primary criterion is not 
satisfied and processing transfers from PC true step 956A 
to different step 958A. 
10 In different step 958A, field 903A is flagged as a 

different field. This effectively terminates 
process 950A. Nevertheless, as indicated above, the 
secondary criterion is evaluated and found not to be 
satisfied. 

15 Next, fields 903B and 904B in the telecine video 

sequence are the same. Thus, each strip activity in strip 
activities SA68 is approximately zero and field 
activity FA68 is also approximately zero. Thus, in strip 
activity test 952B, which compares each of the strip 

20 activities in SA68 to strip activity threshold T_m, none 
of the strip activities exceed the threshold. Similarly, 
in field activity test 954B, which compares field 
activity FA68 to field activity threshold T_n, field 
activity FA68 is less than threshold T_n. 

25 PC true step 9 5 SB checks to determine whether two or 

more strip activities exceed threshold T_m in step 952B or 
field activity FA68 exceeded threshold T_n. Since neither 
threshold was exceeded, the primary criterion is satisfied 
and processing transfers from PC true step 956 to even 

30 parity check step 960B. Since field 903B and 904B are 
even parity, even parity check step 960B transfers to 
repeated field step 962B. Repeated field step 962B marks 
field 903B as a repeated field in this embodiment. This 
effectively terminates process 950B. Nevertheless, the 

3 5 secondary criteria is evaluated and found to be true. 
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Hence, frame 903 has been processed and field 903A is 
not repeated, but field 903B is repeated. Since the 
primary criterion for odd parity field 903A was not 
satisfied, the first bit shifted into repeated field 
5 register RF is a zero. Since the primary criterion for 
even parity field 901B was satisfied, the second bit 
shifted into repeated field register RF is a one. since 
the secondary criterion was not satisfied for field 903A 
but was satisfied for field 903B, the first bit shifted 
10 ln to odd even compare register OEC is a zero and the 
second bit is a one. The values of strip activity 
threshold T_m and field activity threshold T_n are updated 
as described above, because the primary criterion for 
field 903B was satisfied. 
15 The processing of the remaining fields is similar to 

that described above and each- field is determined to be a 
different field. As explained more completely below 
there are a plurality of telecine combinations . However 
the repeated field detection. process is the same sequence 
20 of steps for each telecine video sequence and reliably 
detects each repeated field. 

in the above examples, since the input sequence was 
known, the repeated field register and the odd-even 
compare register have the same information and appear 
25 redundant. However, as explained more completely below 
the primary criterion, i.e, the information contained in 
repeated field register RF, is sufficient to uniquely 
identify a repeated frame, but the primary criterion may 
fail to distinguish between a telecine sequence and frames 
0 with only small differences. Thus, in a real-time 
analysis with an unknown input video data stream 
information in both repeated field register RF and odd- 
even compare register OEC is needed to reliably 
distinguish between repeat frames and telecine sequences 
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In the above examples, it was assumed that each strip 
activity and each field activity was approximately zero. 
However, noise is almost invariably introduced in input 
video data stream 3 01. Thus, it is rare to have either a 
5 strip activity or a field activity that is equal to zero 
even if a field is repeated. In reality, each strip 
activity SA and field activity FA for a pair of repeated 
fields of the same parity should be "close" to zero. In 
the process of this invention, strip activity 

10 threshold T_m and field activity threshold T_n define, 
adaptively, what "close" means. Thus, if a multiplicity 
of strip activities SA and field activity FA are less than 
thresholds T_m and T_n, respectively, the two fields are 
declared identical. 

15 The values chosen for thresholds T_m and T_n and the 

way these thresholds are adapted were empirically 
determined based on computer simulations. Preferably, the 
thresholds should not be too small, or too large. In the 
former case, some repeated fields may not be detected 

20 because the noise may exceed the threshold, while in the 
latter case different fields may be flagged as repeated, 
since the threshold may actually exceed their difference. 
Consequently, the thresholds are clamped within a certain 
range of values that were also selected empirically. 

25 The default values of 4*N_f and 6*N_s for field 

activity threshold T_n and strip activity threshold T_m 
respectively translate to absolute per pixel values of 4 
and 6. As explained above, the. circuitry sums the 
differences between pixel pairs to obtain the, strip and 

30 field activities. If the resultant values are normalized 
by the number of pixels in each, i.e., by the number of 
pixels in a strip N_s and the number of pixels in a 
field N_f, respectively, the result is the average 
pixel-to-pixel difference over a strip and a field. The 

35 default threshold values now imply that at power- up, a 



WO 95/15659 

PCT/US94/13287 

- 78 - 

field is declared repeated if its field-normalized field 
ac tl v lt y value (i.e., per pixel value) is i ess than w . 
The^default values were chosen empirically. Likewise, the 
condition . that two or- more strip activities SA should 
exceed strip activity threshold T_m also was chosen 
empirically. 

Herein, the secondary criterion is a decision based 
on relative" measurements. if the primary criterion is 

10 repeated, or there is only a slight change between two 
successive frames. However, while the primary criterion 
xs sufficient to identify a repeated field, the primary 
criterion by itself is not sufficient to determine if The 
video sequence is a telecine video sequence. The 

" onrr^r 0 ° f 3 t6leCine Vid6 ° SSgUenCe is only 
one fi eld f a frame is repeated _ CQns J 

differentiation between a telecine video sequence and a 
video sequence with a repeated frame, the secondary 

criterion is utilized. 

20 the /* i r Pl "t Md abOVS ' ^ SeCOndarv -iterion compares 
the field activity for a field of a first parity with the 
scene-cut threshold of the other parity field. This 
threshold is a running average of the field activities of 

2s tL o^r r ity - Normaiiy - the scene - cut 

the odd and even parity fields are close to each other, 
bee the field activities of the odd and even parity 

fields are themselves close to each other. The only times 
the field activities could be different are if one If tL 

3 0 I T *T t±eS ±S aSSOciated 1th a repeated field 
30 and/or there is . scen e cut. Thus, by comparing the field 
activity FA of the current field with the scene cut 
threshold of the other parity field, sudden deviations in 

curreL f -^r"^ "* S ^ Cifi ~"V. if the 

current field is repeated, and the other field of this 
frame is not repeated, then the field activity of the 
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current field is likely to be much smaller than the scene 
cut threshold of the other field. Therefore, the 
secondary criterion distinguishes between a sequence of 
fields that appears to be repeat frames but in actuality- 
5 contains a telecine sequence. 

In the previous discussion of state machine 500, 
several conditions were briefly described that determined 
the various transitions between states . The various 
conditions are defined using the information in repeated 

10 field register RF, odd-even compare register OEC, scene 
cut register SC, scan pattern register SP. Table 1 lists 
eleven status conditions, for one embodiment of this 
invention. The status conditions are specified using the 
reference numeral for the register and the conventions of 

15 the C programming language. Also, each status condition 
has two states, a true state and a false state. Sometimes 
herein, the true state is referred to as a status 
condition being satisfied and the false state is referred 
to as a status condition being unsatisfied. 

20 TABLE 1 

Status Conditions 

Condition Explanation 

CO (RF & 0x03 == 0x02) || ( (RF & 0x03 « 0x03) 
&& (OEC & 0x02) ) 

25 CI (RF & 0x01) 

C2 (SC & 0x03 =« 0x03) 

C2' (Scan fit 0x03 == 0x03) 

C3 (SC & 0x06 « 0x06) 

C3' (Scan & OxOC == OxOC) 

30 C4 (SC & OxOC == OxOC) 

C4' (Scan & 0x06 == 0x06) 

C5 (SC & 0x3C == 0x18) 
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C6 (SC & 0x30 == 0x30) 

C6' (Scan & 0x18 == 0x18) 
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07 (SC & 0x60 ==0x60) 

C8 (((R F & 0X04) !( RF & 0x08)) || ( ( ( RF & 

5 0x0C) — OxOC) && (OEC & 0x04))) 

C9 ((RF & 0X03 == 0X01) || ( (RF & 0x03 == 

0x03) && (OEC & 0x01) ) ) 
CIO ((rf & oxCO == 0X80) || ( (RF & OxCO == 

OxCO) && (OEC Sc 0x80) ) ) 
10 011 ( (RF & 0x03 == 0X03) (OEC & 0x03)) 

Each of status conditions CO to Cli are briefly 
considered and then the operation of state machine 480 is 
described in more detail. As explained above, all of 

15 In 9 TatriV XCePt ^ ^ SC ™ 

11 Table X ' are u Pdated at frame boundaries. Thus the 

two least significant bits of the registers correspond to 
the two most recently entered fields. m other words, the 
two least significant bits in each register correspond to 

20 thaT'T ^ ° dd " eVen ° f tW ° f±eldS ° f the *»™ 

20 that most recently entered reordering memory 403 As 

successive frames enter reordering memory 403, the bits in 
these registers are shifted from the least significant bit 
of the register towards the most significant bit of the 
register. J \. t * ,e 

25 when status condition CO is true, it means that the 

. primary criterion for the odd parity field of the most 

recently entered frame is true and the primary criterion. 
, for the even parity field of most recently entered frame 

is false, or the primary criterion for both fields of the 
3 0 frame are true and the secondary criterion for the odd 

ITT f ^ ld " trUS - ThUS ' St3tUS «"»"tion CO is true 
xf the odd parity field in the frame is repeated, but not 
the even parity field. For example, status condition CO 
is true if frame ala2 just entered reordering memory 403 
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and the following frame is alb2 . However, if two 
successive frames are identical, as in a still scene i.e., 
frame ala2 followed .by frame ala2, status condition CO is 
not true . 

5 Status condition CI is true if the primary criterion 

for the even parity field of most recently entered frame, 
i.e., the current frame, is true. 

Notice that in the embodiment given in Figure 5B 
where only the primary criterion is used to determine 

10 whether an even parity field is repeated, i.e., status 
condition CI is true, there are two conditions in status 
condition CO for checking whether an odd parity field is, 
repeated and only one condition in status condition CI to 
check whether an even parity field is repeated. The only 

15 time controller 402 checks for a repeated even parity 
field is after controller 402 has already encountered a 
repeated odd parity field. As discussed above, the 
primary criterion is one indicator of a repeated field, 
but not always a sure indicator. However, the fact that a 

2 0 repeated odd parity field has already been detected at an 
appropriate location in the past provides additional 
information so that finding the primary criterion 
satisfied for the even parity field is considered 
sufficient in this embodiment. In another embodiment, the 

2 5 same status conditions could be used for both even and odd 

parity fields, i.e, status conditions CO and C9 are. used 
to detect repeated fields. 

Status condition C2 is true when the scene cut 
conditions discussed above are satisfied for both fields 

3 0 of the current frame. When status condition C2 is true, 

the current frame is the last frame of the old scene. 

Status condition C2 1 is true, if the current frame 
and the frame formed by taking the odd parity field of the 
current frame and the even parity field of the preceding 
3 5 frame are both interlaced. Status condition C2 • is used 
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to detect where an edit (if any) has occurred As 
explained below, two fields belonging to different frames 
in the input sequence are combined only if the new frame 
produced by combining the two fields is non-interlaced- 
5 otherwise, the combination is not made. 

Status condition C3 is true if the odd and even 
parity fields of the current frame are from different 
scenes. status condition C3 is true when there is a 
m^-frame edit in current frame. status condition C3 < is 
true, if the frame previous to the current frame and the 
frame formed by taking the odd parity field of the frame 
previous to the current frame and the even parity field of 
the frame preceding the frame previous to the current 
frame are both interlaced. 
15 _ Status condition C4 is true if the current frame and 
the one before it belong to different scenes. Hence 
status condition C4 is true when a frame -boundary edit 

fraTs e t ^ bSf0re CUrrSnt framS ' the — 

frame starts a new scene. Status condition C4 ' is true if 

the frame preceding the current frame and the frame formed 

L T ° dd Parlty fi6ld «*■«*• -rrent frame and 

the preceding even parity field are. both interlaced 

Status condition C5 is true if the two fields of the 

frame previous to the current frame belong to different 

25 scenes. status condition C5 is true if there was an 

isolated mid-frame edit in the frame previous to current 

!rrr «* be « . dit 

in. the f le lds immediately preceding and following the 
current frame. m the stream of video data 

30 ala2 blb2 clu2 vlv2 wlw2 

there is a mid-frame edit in frame clU 2 . However, in the 
stream of video data 



20 
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alm2 plx2 



neither frame is flagged as having a mid- frame edit 
because it is likely that such a sequence was purposely 
created and so should not be altered. In another 
5 embodiment, the requirement that a mid- frame edit is 
isolated may not be implemented. 

Status condition C6 is true if there is a scene cut 
between the frame previous to the current frame and the 
frame preceding that frame. For example, the video 

10 sequence, clc2 ulu2 xlx2 yly2, would make status 

condition C6 true where frame xlx2 is the current frame; 
frame- ulu2 is the frame previous to the current 
frame xlx2 ; frame clc2 is the frame preceding frame ulu2 ; 
and there is a scene cut between frame clc2 and 

15 frame ulu2 . Status condition C6' is true if the frame 

previous to the frame preceding the current frame and the 
frame formed by combining the odd parity field of the 
frame previous to the current frame and the preceding even 
parity field are both interlaced. 

20 Status condition C7 is true if the two fields of the 

frame appearing two frames before the current are from 
different scenes. For example, status condition C7 is 
true for the video sequence, clt2, ulu2, xlx2 where 
frame xlx2 is the current frame and frames ulu2 and clt2 

25 are in reordering memory 403. Specifically, status 

condition C7 is true when a mid- frame edit occurred in the 
frame two frames before current frame. 

Status condition C8 is true if the even parity field 
of the frame previous to the current frame is a repeated 

3 0 field. Notice that both the primary and secondary 
criteria are used for status condition C8 . 

Status condition C9 is true if the even parity field 
of the current frame is repeated. Notice that both the 
primary and. secondary criteria also are used for status 
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condition C9 in a manner parallel to that described above 
for status condition CO. 

Status condition CIO is true if the odd parity field 
in the frame that entered reordering memory 403 three 
5 frames before the current frame was a repeated field. 

Again, both the primary and secondary criteria are used in 
the determination. For example, for the video sequence 
ala2 alb2 blc2 clc2 dld2 where frame clc2 is the current 
frame and frame aia2 is the frame that entered reordering 
10 memory 403 three frames before the current field, field al 
is a repeated field and so status condition CIO is true 

Status condition Cll is true when both the even and 
odd parity fields of the current frame are repeated i e 
the current frame is repeated and the secondary criterion' 
IS for both fields is false. 

As explained more completely below, various 
combinations of the status conditions are checked in each 
sub-state of state machine 480 and depending on what 
combination of the status conditions is true 
2 0 controller 4 02 chooses the output format, display code 

and reordering memory read addresses. Consequently, prior 
to considering the detailed operation of state 
machine 480, the output formats and display codes are 
considered. 

25 two read address registers, i.e, read_l register and 

read_2 register, are used to select the data in reordering 
memory 403 that is provided in output video sequence 302 
The two read address registers are loaded, with the address 
of the fields to be read out during the following field 

3 0 period. Hence, when a new address is loaded into either 
or both of the two read address registers, the address 
comes into effect at the first field sync following 
loading of the registers . Read address registers read 1 
register and read_2 register are registers 2010, 2020 ~ 
35 (Figure 20) . 
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In a default mode, controller 4 02 loads new addresses 
into the two read address registers once on every other 
field sync, i.e, every two field periods. However, there 
are situations when a new address is loaded after only one 
5 field period following the last update. To keep track of 
when to load the read address registers, controller 
maintains modulo three output counter OC. During 
power-up, or reset, modulo three output counter OC is 
initialized with a value of two. At each field sync, if 

10 modulo three counter OC has a value greater than zero, 
modulo three output counter OC is decremented by one . 
This is the first thing controller 4 02 does upon seeing a 
field sync. The read address registers are updated then 
if and only if the value of modulo three output counter OC 

15 equals zero. The new address comes into affect at the 
next field sync. Along with Updating the address read 
registers, controller- 402 also resets the value of modulo 
three output counter OC to either one or two. 

As mentioned before, reordering memory 403 has eight 

20 field memory banks, which for convenience are numbered 
consecutively from zero through seven. At power-up, or 
reset, a write pointer is initialized to point to the 
first field location of reordering memory 4 03, i.e., 
location zero. Read_l register is initialized to point to 

25 the third field location and read- 2 register to the fourth 
field location, i.e., the value of read_l register is two, 
and the value of read_2 register is three. At each field 
sync, the write pointer is incremented by one based on 
modulo eight . The read registers are updated by 

30 controller 402 as dictated by state machine 480. 

Apart from updating the read address registers, state 
machine 480 also determines the order in which to read out 
the fields, i.e, the field addressed by read 1 register 
first or the field addressed by read_2 register first, or 
35 if either field should not be read at all. There are five 
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possible output field selection codes in this embodiment 
that are listed, along with the symbolic code for each 
selection, in Table 2. These codes are selected based on 
some combination of ■ the status conditions listed in 
Table 1, as discussed more completely below. 



15 



25 



TABLE 2 

List of output field selection codes 



Output 
Selection 



10 code 
TO 



Tl 



T2 



20 T3 



T4 



Explanation 
read_l register = (read_l register + 2) % 8 
read__2 register = (read_2 register + 2) % 8 

read (field l, field 2) 
read_l register = (read_l register +.2) % 8 
read_2 register = (read_2 register + 2) % 8 

read (field 2, field l) . 
read_l register = (read_l register +2) % 8 
read_2 register = (read_2 register +.2) % s 

read (field 2, field 2) 
read_l register - (read__l register + 2) % 
read_2 register = (read_2 register +■ 2) % 8 

. read (field 1, field, l) 
read_l register = (read_l register + 1) % 8 
read_2 register = (read_2 register + l) % 8 
No data read out 



8 



Notice that for both output field selection codes T2 
and T3, a field is dropped in the frame and one field is 
used for both fields of the frame. 

The nominal delay time introduced by video data 
30 stream analyzer 300 in the video data path is equal to 

eight field periods, i.e, four frame periods. This means, 



WO 95/15659 



PCT/US94/13287 



- 87 - 

in the default mode of operation that corresponds to a 
"clean" video sequence with no mixed-frames, a frame 
entering video data stream analyzer 3 00 exits exactly four 
frame periods later; with its field ordering retained. In 
5 this mode of operation, the reordering memory read and 
write address registers increment at the same rate 
relative to each other. However, when the input video 
data stream is a film source, some perturbations are 
periodically introduced in the read address registers to 

10 eliminate repeated fields. These perturbations could be 
selecting a field that is out of its natural order or not 
reading one of the fields of a frame, or inhibiting the 
output for a field period. Usually, if one field of a 
frame is inhibited, the other field is read out twice to 

15 complete the frame of data. Similarly, in the case of 15 
fps cartoon sequences, the output from reordering 
memory 403 is inhibited during alternate frame periods. 
To facilitate buffer management in the encoder and 
decoder, video data stream analyzer 30 0 introduces dead 

20 times at appropriate intervals in the video output 
sequence, as explained above. 

Along with the output field selection code of 
Table 2, state machine 480 generates a display code. To 
ensure proper display of the fields selected by the output 

25 field selection code in decoding system 120, the display 
code should accompany the two fields selected. As 
mentioned before, this information is part of the side 
channel provided for transmission to decoding system 120. 
Table 3 gives an example of one embodiment of the display 

3 0 codes. 

TABLE 3 
Field display codes 



35 



Display Code 
D02 



Explanation 
display (odd, even) 
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display (odd, even, odd) 
display (even, odd) 
display (even, odd, even) 
display (odd, even, odd, even) 



5 Most display monitors are interlaced. An interlaced 

display monitor does not display frames but rather fields 
at a rate of 60 times a second. In the normal mode, a 3 0 ' 
frames/second video sequence is displayed as 60 
fields/second. That is each frame is displayed for 
10 exactly two field times, i.e., the odd parity field first 
followed by the even parity field. As explained above, 
video data stream analyzer 300 of this invention drops all 
repeated fields that are found, and in general, the fields 
that are dropped are not guaranteed to follow any fixed 
15 pattern, because of edits, etc. m other words, the video 
data output stream from video data stream analyzer 300 
that is compressed and transmitted is not guaranteed to be 
of any fixed frame rate, and is of a rate less than or 
equal to 30 fps. However, the ultimate display device (tv 
20 monitor, e.g.) has to display the uncompressed information 
exactly at the rate of 60 fields/second. 

Hence, the display code tells decoding system 120 how 
to generate a data rate of 60 fields/second. 
Specifically, this is accomplished by displaying some 
25 frames for more than two field times, specifically three 
or even four field times; Display code D02 is the default 
mode for a frame and is the normal interlaced pattern 
Display code D03 indicates that the decoder should display 
a frame for three field periods and specifically, the odd 
30 parity field should be displayed, followed by the even 
parity field, and then the odd parity field again 
Display code D12 indicates that the decoding system should 
display the frame for two field periods, but the even 
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parity field should be displayed first and followed by the * 
odd parity field. Display code D13 is similar to display 
code D03, but the odd and even parity fields are reversed. 
Display code D04 indicates that the decoding system should 
5 display the frame for four field periods in an odd, even, 
repeat odd, repeat even sequence. 

For example, if a cartoon sequence is converted by 
video data stream analyzer 300 from 30 fps to 15 fps and 
then coded and transmitted, display code D04 would be 
10 associated with each frame to tell the decoding system to 
repeat each frame exactly once so as to display on a 60 
fields/second monitor. In this embodiment, the display 
code is loaded in a register that can be read by 
encoder 103 . The particular method used by encoder 103 to 
15 encode the side channel information for transmission to 
decoding system 120 is beyond the scope of this invention. 
The important aspect is that information is provided to 
encoder 103 to enable reconstruction of a 30 frame per 
second video sequence by decoding system 120. 
20 Output video data stream 302 from video data stream 

analyzer 300 can be characterized either as a frame or a 
pair of fields. Frame output means that the alternate 
lines in output video sequence 302 belong to the same 
field while field output means that all lines of field one 
25 in output video sequence 302 are output first followed by 
all the lines of field two. Additionally, production of 
frame output implies that subsequent processing, such as 
vertical resampling would generate visually superior 
images if performed on the frame as a whole. Production 
30 of field output implies that subsequent processing would 
generate visually superior images if performed on the 
fields individually. Alternatively, if it is required 
that the format of the output data be fixed (to either 
frame or field) , video data stream analyzer 300 outputs ■ 
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the scan pattern information as additional side channel 
information. 

A variety of criteria are utilized to select the 
output format. In some cases, the output format is 
5 determined fay the present state of state machine 480, 
e.g., if state machine is in movie state, the output 
format is in frame output format. 

However, in other situations, the state of state 
machine 480 is not sufficient to select the output format 
10 For example, in video state 500, input video data 

stream 301 can be 30 fps interlaced or progressive. In 
this situation, the information in scan pattern register 
SP is used to determine the output format. if bit 3 
where bit 0 is the least significant bit, of scan pattern 
15 register SP is a one, the output format during the next 
frame period is chosen as the field format, and otherwise 
the output format is chosen as the frame format. In the 
subsequent discussion of state machine 4 80, the following 
notation is used for the output format codes- 

20 

TABLE 4 
Output format codes 

Output Format code Explanation 

F0 Output is in frame format . 

25 F1 Output is in field format. 

In the . state machine discussion that follows, when the 
state of bit three in scan pattern register SP determines 
the output format, the output format code is preceded by 
an -I:.- i n the state diagrams. If no -i- appears before 
3 0 the output format code, its selection is assumed not to be 
conditioned on any event. Also, if read code T4 is 
chosen, i.e., no data is read out of reordering 
memory 403, no side channel information is generated The 
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side channel information, i.e, the display code and the 
scan pattern code, is generated only when data is output 
from video data stream analyzer 300. 

As explained above, the default mode of operation of 
5 video data stream analyzer 3 00 is defined by video 
state 500 of state machine 490. Video state 500 is 
illustrated in more detail in Figure 10. Video state 500 
has two sub-states 1001 and 1002. Controller 402 enters 
sub- state 10 01 upon a power-up or reset with modulo three 
10 output counter set to a value of two. The operations of 
seek A state 510, movie state 520, seek B state 530, trans 
state 54 0, and cartoon state 550 are discussed more 
completely below. 

Sub- state 10 01 is executed while an odd parity field 
15 is entering reordering memory 403. Sub-state 1002 is 

entered on the field sync indicating that the even parity 
field is entering reordering memory 4 03 . 

In sub-state 1001, state machine performs the 
operations indicated by Table 5, if and only if modulo 
2 0 three output counter OC has a value zero. 

TABLE 5 

Operations in sub- state 1001 

«if ( C10 && !( 'C6 || C6' | | C7 ) ) : T3 else T4 >> 
<<if ( C10 && ! ( CG || C6' j | C7 ) ) : 
25 OC 2 else OC = 1 » 

<<D13>> 

« if (T3 ||I) : Fl else F0» 

Status condition C10 is required to recover from a 
possible false alarm that might have previously caused 
30 controller 402 to assume that a telecine sequence was 
encountered. Upon detection of the possible start of a 
new telecine sequence, controller 402 transitions from 
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video sub-state 1002 to seek A state ?m u 

«_ ^. state 510. However, if the 

e ;;;7 f the possibie ^ ^ • fw.. se^ wa 

a false alarm, controller 402 loops back to video 
state 500 from seek A state 510. Under these 
5 circumstances, controller 402 has to remix another frame 
to ..undo* the field dropping done in response to the faLe 
alarm. status condition C10 tells controller 402 to take 
these actions. 

If status condition C10 is true and none of status 
10 condxtxons «. and C7 are true, the read registers 

are each advanced by two locations in reordering 
memory 403 by selecting output field selection code 
Field one xs read as both the odd and even parity fields 
of the output frame. One of status conditions ce, C S 

slat " ^ " CO ~ es ^ d -* ^atus condition, i' e 

status condition C2, C2<, or C3, respectively, was ^e 
three sub-states earlier. Thre e sub-states earlier "he 
display code was set to D04 and therefore it is necessary 
to ensure that exactly two dead periods follow. Thxs ls 
20 a- lshed by setting ^ ^ fieid ^ 

TrZ My ° f StatUS C ° nditions C6 ' and C7 are 

field^ri^ iS / ein9 SelSCted f ° r ° UtpUt if the 
field selection code is set to T3, modulo three output 

Z2T:° C 1S reS6t t0 3 ValUe ° f tW ° oth,rwise P to a 

.frame, dxsplay code D13 is selected to indicate that the 
t even; parity field in the frame should be repeated to 

reconstruct the video sequence. if either output field 
30 s; ectxon code T3 was selected or bit 3 in register is 

. ;i; Lr format code is fi and 

eith/^r 11 ' WhSneVer the out P ut selection code is 

35 L b " ° UtPUt C ° de F1 " Sel6Ct ^- T^. 

35 x. because output field selection codes T2 and T3 both 
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require that the same field be read twice and in such a 
situation vertical filtering on a field grid is preferred. 

If modulo three output counter OC is zero and status 
condition CIO is not true, output field selection code T4 
5 is selected and modulo three output counter OC is set to a 
value of one. Output field selection code T4 causes the 
read address registers to be incremented by one location 
and no data is read from reordering memory 4 03. 
Consequently, the display code and the output format are 
10 not utilized and no side channel information is generated. 
On the next field sync, modulo three output 
counter OC is decremented and processing transfers to sub- 
state 1002. Also, if status condition CIO was true, the 
field addressed by read_l register is output twice in 
15 video data output stream 302. 

In sub-state 1002, state machine 480 performs the 
operations indicated by Table 6, if and only if modulo 
three output counter OC has a value zero. 

TABLE 6 

20 Operations in sub-state 1002 

<<if C5 : T2 else if C8 : T3 else TO >> 

<<OC =2>> 

<<D02>> 

«if (T2 | | T3 | | I) : Fl else F0>> 

25 If status condition C5 is true, i.e, a mid-frame edit 

was detected in the previous frame, the read registers are 
each advanced two locations in reordering memory 4 03 by 
selecting output field selection code T2 and the mid- frame 
edit repair is completed by selecting field two as both 

30 the odd and even parity fields of the output frame. If 

status condition C8 is true, i.e, the even .parity field of 
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the frame previous to the current frame was repeated, and 
status condition C5 is not true, the read registers are 
each advanced two locations in reordering memory 403 by 
selecting output field selection code T3 and the repeated 
5 even parity field is eliminated by selecting field one as 
both the odd and even parity fields of the output frame 
If neither status condition C8 is true, nor status 
condition C5 is true, the read registers are each advanced 
two locations in reordering memory 403 by selecting output 
10 code TO and video output sequence rs taken in the normal 
order of field one followed by field two. 

After the output field selection code is determined 
m sub-state 1002, modulo three output counter OC is set 
to a value of two and the default display code D02 is 
15 selected. Finally, if output field selection code T2 
or T3 was selected, or bit 3 in register SP is set, the 
output format code is Fl and otherwise the output format 

If neither status condition CO nor status 
20 condition Cll is true, processing transfers from sub- 
state 1002 to sub-state 1001 on the next field sync. As 
used herein in the discussion of state machine 480, -the 
next field sync- means the first field sync that occurs 
25 Hill sub -"ate entered. Consequently, as long as 

25 neither status condition CO nor status condition Cll is 
satisfied, state machine 480 and consequently 

controller 4 02 continues in video state 500. Thus if 
r ; input video data stream 301 is a regular 30 f ps video 
sequence, controller 402 continues outputting the same 
30 sequence, unaltered. If a mid-frame edit is detected in 
input video data stream 301, the first field of the 
mixed-frame is replaced with the second field. 

If either status condition CO or status condition Cll 
is true, processing does not return from sub-state 1002 to 
35 sub-state 1001,. but rather branches to a new state on a 
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field sync. If status condition CO is true, processing 
transfers from sub-state 1002 in video state 500 to sub- 
state 1101 (Figure 11) in seek A state 510. State 
machine 4 80 also enters sub- state 1101 from trans 
5 state 540 and movie state 520. As shown in Figure 11, 
seek A state 510 has four sub-states 1101, 1102, 1103, 
and 1104 . Since seek A state 510 is entered whenever a 
telecine sequence is expected, the four additional field 
periods of seek A 510 are needed to ascertain whether 
10 input video data stream 301 is actually a telecine 
sequence . 

Sub-states 1101 is executed while a first odd parity 
field is entering reordering memory 403. Sub-state 1102 
is entered on the field sync indicating that a first even 

15 parity field is entering reordering memory 403. 

Sub-state 1103 is entered on the field sync indicating 
that a second odd parity field is entering reordering 
memory 403. Sub-state 1104 is entered on the field sync 
indicating that a second even parity field is entering 

20 reordering memory 403. Note that the reference to a field 
entering reordering memory 4 03 is made with respect to 
entering seek A state 510 and not with respect to an 
absolute measure of the number of fields entering 
reordering memory 4 03. 

2 5 in sub-state 1101, if modulo three output counter OC 

has a value of zero, the output field selection code is 
set to T4 and modulo three output counter OC is set to 
have a value of one. See Table 7. Consequently, the read 
address registers are each incremented by one location and 

3 0 no data is read from reordering memory 403. On the next 

field -sync, processing transfers from sub-state 1101 to 
sub- state 1102 and modulo three output counter OC is 
decremented . 
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TABLE 7 

Operations in sub- state 1101 



<<T4>> 
<<OC=l>> 



5 In sub-state 1102, if modulo three output counter OC 

has a value of zero, the operations given in Table 8 are 
performed . 



TABLE 8 

Operations in sub-state 1102 

10 <<if C5 : T2 else T0» 

<<OC=2>> 

«if (C2 || C2' || C3) : D04 else D03» 
<=<if ( T2 | | I) : Fl else F0» 

Specifically, if status condition C5 is true, i.e, a mid- 
15 frame edit was detected in the previous field, the read 
registers are each advanced two locations in reordering 
memory 403 by selecting output field selection code T2 and 
the mid-frame edit repair is completed by selecting field 
two as both the odd and even parity fields of the output 
20 frame. If status condition C5 is not true, "the read 
registers are each advanced two locations in reordering 
memory 403 by selecting output code TO and the video 
output sequence is taken in the normal order of field one 
followed by field two. 
25 • After the output field selection code is determined 
an sub-state 1102, modulo three output counter OC is set 
to a value of two. If either status condition C2, status 
condition C2' or status condition C3 is true, display 
code D04 is selected and otherwise, display code D03 is 
3 0 selected. As indicated above, display code D04 means that 
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the frame is repeated while display code D03 means that 
the odd parity field in the output frame is repeated. 
Finally, if output field selection code T2 was selected, 
or bit 3 in register SP is set, the output format code 
5 is Fl and otherwise the output format is FO . 

On the next field sync, processing transfers from 
sub-state 1102 to sub- state 1103 and modulo three output 
counter OC is decremented. Since modulo three output 
counter OC can not take on a zero value in sub-state 1103, 
10 state machine 480 is quiescent during this field period. 
On the next field sync, processing transfers from sub- 
state 1103 to sub-state 1104. 

State machine 480 performs the operations in Table 7 
upon entering sub- state 1104 if modulo three output 
15 counter OC has a value of zero. Since these operations 
are identical to those given in sub-state 1101, the 
description given of these operations for sub- state 1101 
is incorporated herein by reference. 

As mentioned above, controller 402 enters this state 
20 whenever a first repeated odd parity field is detected 
where "first repeated odd parity field" means the first 
repeated odd parity field in a new telecine sequence. At 
the end of sub-state 1104 , if controller 402 sees a 
repeated even parity field, i.e, status condition C9 is 
25 true, a telecine sequence has been detected. Note that in 
the telecine sequence the repeated even parity field 
appears five fields after the repeated odd parity field, 
however, one field interval out of the five was passed in 
video state 500. Thus, at the end of sub-state 1104, seek 
30 A state 510 transfers to either video state 500 or movie 
state 520 depending on the state of status condition C9 . 

If status condition C9 is true, a telecine sequence 
has been encountered and so processing transfers on the 
next field sync from sub-state 1104 in seek A state 510 to 
35 sub-state 1201 (Figure 12) in movie state 520. Also, 
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modulo three output counter OC is decremented. Since a " 
telecine sequence has been detected, movie state 520 has 
six sub-states 1201 to 1206, because another repeated 
field xs not anticipated for five fields, which is the 
first field of a frame. 

In sub-state 1201, if modulo three output counter OC 
has a value of zero, the operations given in Table 9 are 
performed . 



TABLE 9 

Operations in sub-state 12 01 
«if ( C6 J! C6' || C7 ) : T4 else Tl» 
«if C C6 || C6' I I C7 ) : OC-1 else OC=2» 
<<D12>> 

<<F0>> 



15 Specifically, if any one of status condition C6, C6' or C7 
is true, output field selection code T4 . The read 
registers are each advanced one location in reordering 
memory 403 by selecting output field selection code T4 and 
the subsequent selection of the display code and the 
20 output format are ignored because no data is read from 
reordering memory 403. As explained above, status 
condition C6 is true if there is a scene cut between the 
frame previous to the current frame and the frame 
preceding that frame. Status condition C6' is true if the 
frame previous to the frame preceding the current frame 
and the frame formed by combining the odd parity field of 
the frame previous to the current frame and the preceding 
even parity field are both interlaced. • status condition 
C7 » true if the frame preceding the current frame by two 
frame had a mid-frame edit. if all of status conditions 
«, C6' and C7 are not true, the read registers are each 
advanced two locations in reordering memory 403 by 
selecting output code Tl and the video output sequence is 
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taken in the inverse of the normal order, i.e., field two 
followed by field one. 

After the output field selection code is determined 
in sub- state 1201, if any one of status condition C6, C6 ' 
5 or C7 is true, modulo three output counter 0C is set to a 
value of one and otherwise to a value of two and display 
code D12 is selected. As indicated above, display 
code D12 means that the even parity field in the frame is 
displayed and then the odd parity field. Finally, bit 

10 three of scan pattern register SP is not used to select 
the output format, because state machine 480 is in movie 
state 520 and the frame mode is automatically selected as 
the output format . 

On the next field sync, processing transfers from 

15 sub-state 1201 to sub-state 1202 and modulo three output 
counter OC is decremented. In sub-state 1202, if modulo 
three output counter OC has a value of zero, the 
operations given in Table 10 are performed. 

TABLE 10 

20 Operations in sub-state 1202 

<<T3>> 
<<OC=2>> 
<<D04>> 
<<F1>> 

25 Specifically, the output field selection code is set to T3 
which advances the read, registers by two locations in 
reordering memory 4 03 and field one is repeated in the 
video output sequence. Modulo three output counter OC is 
set to a value of two and the display code is set to D04 . 

3 0 As indicated above, display code D04 means that the output 
frame is repeated. Again, bit three of scan pattern, 
register SP is not used to select the output format, 
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because state machine 480 is in movie state 520. On the 
next field sync, processing transfers from sub-state 1202 
to sub-state 1203 and modulo three output counter OC is 
decremented. 

5 in sub-state 1203, controller performs the operations 

given m Table 11 if value of modulo three output 
counter OC is zero. 



TABLE 11 
Operations in sub-state 1203 

10 <<T1>> 

<<OC=2>> 

<<D13>> 

<<F0>> 



Specifically, the output field selection code is set to XI 
15 which advances the read registers by two locations in 
reordering memory 403 and the video output sequence is 
taken in the inverse, of the normal order, i.e., field two 
13 followed ^ "eld one. Modulo three output counter OC 
is set to a value of two' and the display code is set 
to ma. as indicated above, display code D13 means that 
the even parity field in the output frame is displayed, - 
then the odd parity field, and then the even parity field 
again. Again, bit three of scan pattern register SP is 
,L USSd t0 Select the ""Put format, because state 

25 machine 480 is in movie state 520. On the next field 
sync, processing transfers to sub-state 1204 and modulo 
three output counter OC is decremented. 

in sub-state 1204, if modulo three output counter OC 
has a value of zero, the operations in Table 12 are 
30 performed. 
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TABLE 12 
Operations in sub-state 1204 

<<T4>> 
<<0C=1>> . 

5 The output field selection code is set to T4 and 

modulo three output counter OC is set to have a value of 
one. Consequently, the read address registers are each 
incremented by one location and no data is read from 
reordering memory 4 03. 

10 Also in state 1204, status conditions C4 and CO are 

checked. If both status conditions C4 and CO are true, 
processing on the next field sync goes to sub- state 1101 
in seek A state 510 and otherwise to sub-state 1205. 
Conditions CO and C4 are both true when the current frame 

15 and the frame before the current frame belong to different 
scenes and the odd parity field in the current field is 
repeated and the even parity field is not. In other 
words, the previous frame ended one telecine video 
sequence, and the current frame, i.e., the last frame that 

2 0 enter reordering memory 4 03, appears to be the start of 

another telecine video sequence. If either status 
condition C4 or CO is not true,, on the next field sync, 
processing transfers from sub-state 1204 to sub-state 1205 
and modulo three output counter OC is decremented . 
25 in sub-state 1205, modulo three output counter OC has 

a value of zero, and so the output field selection code is 
set to T4 and modulo three output counter OC is set to 
have a value of one. See Table 13.. Consequently, the 
read address registers are each incremented by one 

3 0 location and no data is read from reordering memory 4 03. 

On the next field sync, processing transfers from sub- 
state 1205 to sub- state 1206 and modulo three counter 
output OC is decremented. 
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TABLE 13 
Operations in sub-state 120S 

<<T4>> 
<<OC=l>> 

In sub-state 1206, state machine 480 performs the 
operations indicated by Table 14. 

TABLE 14 

Operations performed in sub- state 1206 



«if C5 : T2 else if .( !C0 && C6 ) : T3 else TO » 
10 «0C =2» 

<<D02>> 

«if (T2 | | T3 | | I) : Fl else F0» 

If status condition C5 is true, i.e, a mid-frame edit 
was detected in the previous field, the read registers are 
15 each advanced two locations in reordering memory 403 by 
selecting output field selection code T2 and the mid-frame 
edit repair is completed by selecting field two as both 
the odd and even parity fields of the output frame. if 
status condition GO is false and status condition C6 is 
2 0 true, i.e, the odd parity field of the current frame is 
not repeated and there was a scene cut between the frame 
previous to the current frame and the frame preceding the 
previous frame, and status condition CS is not true, the 
read registers are each advanced two locations in 
25 reordering memory 403 by selecting output field selection 
code T3 and the repeated or isolated even parity field is 
eliminated by selecting field one as both the odd and even 
parity fields of the frame. If either status condition CO 
is true or status condition C6 is not true, and status 
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condition C5 is not true, the read registers are each 
advanced two locations in reordering memory 4 03 by 
selecting output code TO and video output sequence 302 is 
taken in the normal order of field one followed by field 
5 two . 

After the output field selection code is determined 
in sub- state 120 6, modulo three output counter OC is set 
to a value of two and default display code D02 is 
selected. Finally, if output field selection code T2 
10 or T3 was selected, or bit 3 in register SP is set, the 
output format code is Fl and otherwise the output format 
is F0. Note at this point, the output format code is 
processed because the telecine sequence could have 
actually ended a few frames back and so the expected 
15 repeat of the odd parity field may not be detected. 

On the next field sync, processing transfers to 
either sub-state 1001 in video state 500 or sub-state 1301 
in seek B state 530. Specifically, if status condition CO 
is true, i.e., if the expected repeated odd parity field 
20 has been detected, processing transfers to sub-state 1301 
and otherwise to sub-state 1001. 

As shown in Figure 13, seek B state 530 has four 
sub-states 1301, 1302, 1303, and 1304. Seek B state 530 
is entered whenever a repeated odd parity field of a 
25 telecine sequence has been detected. The four additional 
field periods are needed to ascertain whether the input 
video data stream is still continuing as a telecine 
sequence . 

Sub-state 1301 is executed while a first odd parity 
30 field is entering reordering memory 403. Sub-state 1302 

is entered on the field sync indicating that a first even 

parity field is entering reordering memory 403. 

Sub- state 13 03 is entered on the field sync indicating. 

that a second odd parity field is entering reordering 
35 memory 403. Sub-state 1304 is entered on the field sync 
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indicating that a second even parity field is entering 
reordering memory 403. The reference to a field entering 
reordering memory 403 is made with respect to entering 
seek B state 53 0 and not with respect to an absolute 
5 measure of the number of fields entering reordering 
memory 403 . 

In sub-state 1301, modulo three output counter OC can 
not have a zero value and so no operations are performed 
in this sub-state. On the next field sync, processing 
10 transfers to sub-state 1302 and modulo three output 
counter OC is decremented. 

In sub-state 1302, if modulo three output counter OC 
has a value of zero, the operations given in Table 15 are 
performed. 



15 



TABLE 15 
Operations in sub-state 1302 



<<T0>> 
<<OC=2>> 

«if ( C2 || C2' : || C3 } : D04 else D03» 
20 <<F0>> 

Specifically, the read registers are each advanced two 
locations in reordering, memory 403. by selecting output 
field selection code TO and the video output sequence is 
taken in the normal order of field one followed by field 
25. two. .Modulo three output counter OC is set to a value of 
two. 

If any of one of status condition C2, status 
condition C2<, or status condition C3 is true, the display 
code is set to D04 to indicate that the current frame 
3 0 should be displayed in the normal. order of the odd parity 
field followed by the even parity field and then the 
display should be repeated. if none of status 
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condition C2 , status condition C2', or status condition C3 
is true, the display code is set to D03 . As indicated 
above, display code D03 means that the odd parity field in 
the frame is repeated. Finally, bit three of scan pattern 
5 register SP not is used to select the output format. 

On the next field sync, processing transfers from 
sub- state 13 02 to sub-state 13 03 and modulo three output 
counter OC is decremented. Since modulo three output 
counter OC can not take on a zero value in sub- state 13 03, 

10 state machine 480 is quiescent during this field period. 
On the next field sync, processing transfers from sub- 
state 1303 to sub-state 1304. 

State machine 4 80 performs the operations in Table 16 
upon entering sub- state 13 04 if modulo three output 

15 counter OC has a value of zero. 

TABLE 16 
Operations in sub-state 13 04 

<<T4>> 
<<OC=l>> 

20 Since these operations are identical to those given in 
sub- state 1101, the description given for sub- state 1101 
is incorporated herein by reference. 

As mentioned above, controller 4 02 enters this state 
whenever a repeated odd parity field is detected in movie 

25 state 520. At the end of sub-state 1304, if controller 
sees a repeated even parity field, i.e, status 
condition C9 is true, another telecine sequence has been 
detected. Note that in the telecine sequence the repeated 
even parity field appears five fields after the repeated 

30 odd parity field, however, one field interval in the 

sequence was passed in movie state 520. Thus, at the end 
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of sub- state 1104, seek B state 530 transfers to movie 
state 520 if status condition C9 is true. 

If status condition C9 is not true, state machine 480 
also determines whether status condition CO and any one of 
5 status conditions C3 ' , C4, or C5 is true. if status 
condition CO and any one of status conditions C3 ' , C4, 
or C5 are true, processing transfers to sub-state ' 1101 in 
seek A state 510 and otherwise to sub-state 1401 in trans 
state 540. if status conditions C4 or C5 are true, either 
a mid- frame edit was detected or a scene . change occurred. 
As indicated previously, status condition CO is true 
whenever the start of a telecine sequence is expected. 
Thus, processing transfers to seek A state 510 when the 
previous telecine sequence was interrupted, but it appears 
15 that another telecine sequence is starting. 

If processing does not transfer from sub-state 13 04 
to either movie state 520 or seek A state 510, the 
telecine sequence has apparently ended and so 'processing 
transfers to trans state 54 0 to assure that the sequence 
20 is terminated properly. 

Note that in the embodiment illustrated in Figure SB 
where only the primary criterion is used to detect 
repeated even parity fields, i.e, status condition CI is 
used instead of status condition C9, there is what appears 
25 to be an indeterminate transition. Status condition CI 
bemg true tells controller 402 to transition to movie 
state 520 while condition ((C3< || C4 || C5> && CO) being 
true tells controller 402 to transition to seek A 
state 510. If this situation occurs, controller 402 gives 
3 0 preference to the transition to movie state 52 0. 

Trans state 540 (Figure 14) has six sub-states 1401 
to 1406. Sub-state 1401 is'" entered only from seek B 
state 430. If modulo three output counter OC has a value 
of zero, the operations in Table 17 are performed and 
35 otherwise, on the next field sync, processing transfers 
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from sub-state 1401 to sub-state 1402 and modulo three 
output counter OC is decremented. 

TABLE 17 
Operations in sub-state 1401 

5 << if (C6' | | C6 | | C7 ) : T4 else Tl >> 

« if (C6' | | C6 | | C7 ) : OC = 1 else OC =2 » 

« if (C4 || C4' j| C5 ): D13 else D12 >> 
<< I : Fl else F0 >> 



In sub- state 1401 if modulo three output counter OC 

10 has a value of zero and if any one of status conditions 
C6, C6' or C7 are true, the output field selection code is 
set to T4 and otherwise to Tl . As described above, output 
field selection code T4 suppresses output from reordering 
memory 4 03 and output field selection code Tl outputs a 

15 frame from reordering memory 403 in the normal format of 
the odd parity field followed by the even parity field. 

If any one of status conditions C6, C6 ' or C7 are 
true, modulo three output counter OC is set to a value of 
one and otherwise to a value of two. If any one of status 

20 conditions C4 , C4 ' , or C5 is true, the display code is set 
to D13 and otherwise to D12 . Finally, if bit 3 in 
register SP is set, the output format code is Fl and. 
otherwise the output format is F0 . 

In sub-state 1401, one of status conditions C6, C6', 

25 or C7 is true if the corresponding status condition, i.e., 
status condition C2, C2' , or C3 , respectively, was true 
three sub-states earlier. Three sub-states earlier, the 
display code was set to D04 and therefore it is necessary 
to ensure that exactly two dead periods follow. This is 

3 0 accomplished by setting the output field selection code 
to T4. If none of status conditions C6, C6' or C7 are - 
true, the trailing frames are remixed and hence the output 
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field selection code is Tl. After performance of ^ 
operations in sub-state 140!, on the next field sync 
Processing transfers from sub-state 140! to sub-state 1402 
_ and modulo three output counter OC is decremented. 

in sub-state 1402, the operations given in Table 18 
are performed if mod ulo three output counter OC has a 
value of zero. 



TABLE 18 
Operations in sub-state 1402 



10 « if C8 : T3 else TO » 

«OC =2>> 
<<D02>> 

«if f T3 | | I ) : F1 else F0 



15 % ! tU " C ° ndition C8 is true < i-e. the even parity 

15 fxeld of the frame previous to the current frame was 
repeated, the read registers are each advanced two 
locates in reordering memory 403 by selecting field 
output fxeld selection code T3 and the repeated even 

20 HTjJ'T ±S eliminated * -Acting field one as both 
20 the odd and even parity fields of the output frame. if 
status condxtion C8 is not true, the read registers are 
each advanced two locations in reordering memory 403 by 
selectxng output. field selection code TO and video output 

25 foTr: h° v taken in the normai ° rd - ° f fie " °- 

25 followed by field two. 

After the output field selection code is determined 
xn sub-state 1402, modulo three output counter OC Ts 7e t 
to a value of two and default display code D02 is 

0 x S ; ; Tr*- . Finally ' " ^ ° UtPUt field -Action code 
0 T3 or bxt three of scan pattern register SP i s true 
the output format code is Pi and otherwise F0 
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If status condition CO is not true, processing 
transfers from sub-state 1402 to sub-state 1403 on the 
next field sync. If status condition CO is true, i.e., 
another telecine sequence appears to have started, 
5 processing transfers from sub-state 1402 to seek A 
state 510. 

In sub-state 1403, the operations given in Table 19 
are performed if modulo three output counter OC has a 
value of zero. 

10 TABLE 19 

Operations in sub- state 14 03 

<<if C5 : Tl else if ( C6 || C6' j j C7 ) : 

T4 else T0>> 
«if ( C6 | | C6' || C7 ) : OC = 1 else OC=2>> 
15 <<D13>> 

<<I : Fl else F0>> 

Specifically, if status condition C5 is true, i.e, a mid- 
frame edit was detected in the previous frame, the read 
registers are each advanced two locations in reordering 

20 memory 403 by selecting output field selection code Tl and 
the mid- frame edit repair is completed by output ting the 
fields in the inverse of the normal sequence, i.e., the 
even parity field followed by the odd parity field. If 
either status condition C6 , status condition C6 ' , or 

25 status condition C7 is true, i.e, a scene cut has 

occurred, and status condition C5 is not true, the read 
registers are each advanced one location in reordering 
memory 4 03 by selecting output code T4 and no fields are 
output in video output sequence 3 02. If neither .status 

3 0 condition C6 , status condition C6' , nor status condition 
C7 is true, and status condition C5 is not true, the read 
registers are each advanced two locations in reordering 
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memory 403 by selecting output field selection code TO and 
the fields are output in video output field selection 
sequence 302 in the normal order. 

After the output field selection code is determined 
5 ln sub-state 1403, if either status condition CS, status 
condition C6-, or status condition C7 is true, modulo 
three output counter OC is set to a value of one and 
otherwise to a value of two. The display code is set 
to D13. As indicated above, display code D13 means that 
10 the even parity field should be displayed followed by the 
odd parity field and then the even parity field in the 
frame should be repeated. Finally, bit three of scan 
pattern register SP is use d to select the output format 
On the next field sync, processing transfers from 
15 sub-state 1403 to sub-state 1404 and modulo three output 
counter OC is decremented. if modulo three output 
counter OC takes the value of zero in sub-state 1404, the 
operations given in Table 18 are performed. Specifically 
the operations performed in sub-state 1404 are identical ' 
20 to those performed in sub-state 1402 and that description 
xs incorporated herein by reference. Prior to leaving, 
sub- state 1404, state machine 480 checks status 
condition CO to ascertain whether it appears that another 
telecxne sequence has started. if status condition CO is 
25 true, processing transfers on the next field sync to seek 
state 510 and otherwise to sub-state 1405. 

State machine 480 performs the operations in Table 7 
u P? n Bering sub- state 1405 if modulo three output 
counter OC has a. value of zero. since these operations 
30 are identical to those given in sub-state 1101, the 

description of the operations given for sub-state 1101 is 
incorporated herein by reference, On the next field sync 
processing transfers from sub-state 1405 to sub- 
state 1406. 
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In sub-state 1406, state machine 480 performs the 
operations indicated by Table 2 0 if modulo three output 
counter OC has a value of zero. 



«if C5 : T2 else if ( I CO && C8 ) : T3 else TO » 

«OC =2» 

«D02» 

« if ( T2 | ! T3 ! | I ) : Fl else F0» 

10 If status condition C5 is true, i.e, a mid-frame edit 

was detected in the previous frame, the read registers are 
each advanced two locations in reordering memory 4 03 by 
selecting output field selection code. T2 and the mid-frame 
edit repair is completed by selecting field two as both 

15 the odd and even parity fields of the frame. if status 
condition CO is false and status condition CS.is true, 
i.e. , the odd parity field of the current field is not 
repeated and the even parity field in the frame preceding 
the current frame was repeated, and status condition C5 is 

20 not true, the read registers are each advanced two 

locations in reordering memory 4 03 by selecting output 
field selection code T3 and the repeated even parity field 
is eliminated by selecting field one as both the odd and 
even parity fields of the frame. If either status 

25 condition CO is true or status condition C8 is. not true, 
and status condition C5 is not true, the read registers 
are each advanced two locations in reordering memory 4 03 
by selecting output field selection code TO and video 
output sequence 3 02 is taken in the normal order of field 

3 0 one followed by field two. 

After the output field selection code is determined 
in sub-state 14 06, modulo three output counter OC is set 



TABLE 2 0 
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Operations performed in sub-state 1406 
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to a value of two and default display code D02 is 
selected. Finally, if output field selection code T2 
or T3 was selected, or bit 3 in register SP is set, the 
output format code is Fl and otherwise the output format 
5 is FO . 

On the next field sync, processing transfers to 
either sub-state 1001 in video state 500 or sub-state uoi 
xn seek A state 510. Specifically, if status condition CO 
is true, a . e ., the possible start of another telecine 
10 sequence has been detected, processing transfers to sub- 
state 1101 and otherwise to sub-state 1001. 

Recall that in sub-state 1002 of video state 500 
processing transferred to cartoon state 550 if status 
condition Cll was true. Cartoon state 550 (Figure 15, has 
15 four sub-states 1501 to 1504 and is entered only from 
video state 500. 

State machine 480 performs the operations in Table 7 
upon entering sub-state 1501 if modulo three output 
counter OC has a value of zero. since these operations 
20 are identical to those given in sub-state 1101, the 

description given for the operations of sub-state uoi i s 
incorporated herein by reference. On the next field sync 
processing transfers from sub-state 1501 to sub- 
state 1502. 

25 in sub-state 1502, state machine 480 performs the 

operations given in Table 21, if modulo three output 
counter OC has a value of zero. 

TABLE 21 

Operations performed in sub-state 1502 

30 «if cs : T2 else if C8 , T3 else TO » 

«OC =2>> 

«if ( CO I I C9 I I Cll ) : D02 else D04 » 
« ( T2 | | T3 | | I ) : pi else F0» 
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If status condition C5 is true, i.e, a mid- frame edit was 
detected in the previous frame, the read registers are 
each advanced two locations in reordering memory 403 by 
selecting output field selection code T2 and the mid- frame 
5 edit repair is completed by selecting field two as both 
the odd and even parity fields of the frame. If status 
condition C8 is true, i.e, the even parity field in the 
frame preceding the current frame was repeated, and status 
condition C5 is not true, the read registers are each 

10 advanced two locations in reordering memory 4 03 by 

selecting output field selection code T3 and the repeated 
even parity field is eliminated by selecting field one as 
both the odd and even parity fields of the output frame . 
If status condition C8 is not true and status condition C5 

15 is not true, the read registers are each advanced two 
locations in reordering memory 4 03 by selecting output 
field selection code TO and video output sequence 3 02 is 
taken in the normal order of field one followed by field 
two . 

20, After the output field selection code is determined 

in sub- state 1406, modulo three output counter OC is, set 
to a value of two. If any one of status' conditions CO , C9 
or Cll is true, the output display code is set to D02 and 
otherwise to D04 . Finally, if output field selection code 

25 T2 or T3 was selected, or bit 3 in register SP is set, the 
output format code is Fl and otherwise the output format 
is F0. 

On the next field sync, processing transfers to 
either sub-state 1001 in video state 500, sub-state 1101 
30 in seek A state 510, or sub-state 1503. Specifically, if 
status condition CO is true, i.e., the possible start of 
another telecine sequence has been detected, processing 
transfers to sub-state 1101. If status condition CO is 
not true and either status condition C9 or Cll is true 
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processing transfers to video state 500 and otherwise to 
sub-state 1503. 

In sub-state 1503, modulo three output counter OC can 
not have a zero value and so no operations are given. On 
5 the next field sync, processing transfers from sub- 
state 1503 to sub-state 1504. 

State machine 480 performs the operations in Table 7 
upon entering sub-state 1504 if modulo three output 
counter OC has a value of zero. Since these operations 
10 are identical to those given in sub-state 1101, the 

description of the operations given for sub-state noi is 
incorporated herein by reference. On the next field sync 
processing transfers from sub-state 1504 to one of sub- 
state 1501, video state 500 or seek A state 510. 
15 Specifically, if status condition Cll is true, i.e. 

the cartoon sequence continues, processing transfers to" 
sub-state 1501. If status condition Cll is not true and 
status condition CO is true, processing transfers to 
seek A state 510, and otherwise to video state 500. 
2 0 Figure 4B is a block diagram of one embodiment of 

this invention. Statistical analyzer 401 is shown in more 
detail in Figure 16. In this embodiment, the luminance 
data and chrominance data are supplied to statistical 
analyzer 401 by eight bit bus 1601 and eight bit bus 1602 
25 respectively from buffers. One eight bit bus carries 
luminance data. The other eight bit bus carries samples 
alternating between the associated two chrominance data 
e.g., cb and Cr. The video format with chrominance having 
half the horizontal resolution of luminance is know as 
30 4:2:2. Hence, this hardware embodiment processes video 
data in which the two chrominance data each have half the 
number of pixels per line relative to the luminance data 
Further, the hardware can process any frame size up to and 
including the CCIR-601 size (See ITU-R Recommendation 601 
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that was formally CCIR Recommendation 601) for both PAL 
and NTSC standards. 

Eight bit bus 1601 and eight bit bus 1602 combine to 
form input bus 3 01 which drives field memory 450 and field 
5 memory 450 passes the data to field memory 451. In this 
embodiment, field memories 450, 451 each store all the 
data from each field, including vertical and horizontal 
blanking information. The chrominance data is only passed 
through field memories 450 and 451 and then passed 

10 directly to reordering memory 4 03. Hence, statistical 
analyzer 401 provides two field delays for both 
chrominance and luminance data, where a field delay is the 
time period between fields on input bus 3 01. Note, 
herein, a line and a signal on a line are given the same 

15 reference numeral. 

The luminance data are provided to differential 
accumulators 452 to 454, as described above. In addition, 
differential accumulators 452 to 454 each receive video 
timing signals from statistical analyzer timing 

20 controller 1650. As described above, differential . 
accumulators 452 to 4 54 determine the absolute error 
between the fields of the same parity in adjacent frames, 
between adjacent fields, and between line pairs within the 
same field, respectively, and accumulate the differences 

25 between the luminance samples of the appropriate lines and 
fields. Line memory 455 stores all line values. 

Figure 17A is a more detailed schematic diagram of 
each of differential accumulators 452 to 454. In one 
embodiment, the circuitry shown in Figure 17A is 

3 0 implemented in a fieid programmable gate array sold by 
Altera Corporation of San Jose, CA as Model No. EPM 7096. 
Element 1701 forms the difference between two eight bit 
pixels and the difference is stored in register 1702 . The 
output signals from registers 1702 is provided to 

3 5 element 1703, which takes the absolute value of the 
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difference. The absolute value is loaded in 
regis.ter 1704. The output signal of register 1704 is a 
first set of input signals to summer 1705. A second input 
signal to summer 1705 is provided by accumulator 1706. 
5 When a strip of input data has been processed, the output 
signal of accumulator 1706 is the difference metric, that 
is utilized as described above. 

The remaining circuitry in Fig. 17A provides function 
control and timing control for differential accumulator 
10 1700 and also generates an interrupt after each strip of 
input data is processed. Specifically, input signal FMODE 
controls the function of differential accumulator 1700. 
If input signal FMODE is active, differential accumulator 
1700 sums the differences from all lines in the input data 
15 for the same and opposite polarity field accumulators. If 
input signal FMODE is inactive, differential accumulator 
1700 sums the differences for every other line for the 
intra- field accumulator. 

Input signal VALID* , which is active low, marks the 
20 active pixels in each line of input data. Specifically 
signal VALID* goes active for the first active pixel and 
remains valid through the last active pixel in a line. 
Similarly, signal VSync goes inactive at the start of the 
active lines in the field. The low signals on lines 
25 VALID* and VSync drive the output signal of gate 1730 
active. The active signal from gate 1730 is clocked 
through registers 1710, 1711 and drives a first input 
..terminal, of AND gate 1713. The second input terminal of 
AND gate 1713 is driven by the output signal of OR 
3 0 gate 1712. OR gate 1712 has signal FMODE as a first input 
signal and the signal from terminal Q0 from four bit 
counter 1716 as a second input signal, which is active for 
every other line of data. 

Thus, if signal FMODE is active, the output signal of 
$5 AND gate 1713, which drives load terminal LD of 
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accumulator 1706, goes active two clock periods after 
signal VALID* goes active and loads the output signal of 
summer 1705 in accumulator 1706 and continues to do so for 
each active pixel in the line of input data. Conversely, 
5 if signal FMODE is inactive, the output signal of AND 
gate 1713 is active every other line and consequently 
accumulator 1706 loads difference data for only every 
other line. 

When signal VALID* goes inactive after the last 

10 active pixel in a line, one clock period later, the output 
signal of AND gate 1714 goes high. The high signal from 
AND gate 1714 is clocked through register 1715 to enable 
terminal EN of four bit counter 1716. Thus, four bit 
counter 1716 is incremented by one after each line of 

15 input data. When the sixteenth line of data has been 

processed, four bit counter 1716 rolls over and generates 
an active signal on output terminal RCO. 

The active signal from output terminal RCO and the 
active signal from register 1715 drive the output signal 

20 of AND gate 1717. active. The active output signal from 
AND gate 1717 drives register 172 5 and so one clock period 
later the output signal of register 1725 from the inverted 
output terminal drives the signal on line OUTREG_EN* 
active. The active signal on line OUTREG_EN* loads the 

25 output signal of accumulator 1706 into an output register 
that is external to device 1700. The information in the. 
output register, i.e, the absolute value error 
accumulation, is available to statistical processor 490 
(Figure 4B) during the input time for the next 16 lines. 

30 The active output signal from AND gate 1717 (Figure 

17A) also results in an active signal on terminal CL of 
accumulator 1706 as the output signal from 
accumulator 1706 is loaded into the output register. 
Further, the active output signal of AND gate 1717 results 

35 in the generation of an inactive signal on accumulator 
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interrupt line MAE IRQ one clock period after the signal on 
line OUTRE G_EN * goes active. On the subsequent clock 
pulse the signal on accumulator interrupt line MAE IRQ is 
driven high and the rising edge notifies statistical 
5 processor 4 90 that difference data is ready in the output 
register. 

Counter 1716 and accumulator 1706 are cleared at 
power-up and a board reset by reset signal RESET* from 
statistical processor 490. Counter 1716 and 
10 accumulator 1706 are also cleared at the start of each 
field. Specifically, Figure 17B illustrates the timing 
relationship of signals VSync, FIELD, FIELDRST*. Time 
period 1760 is one frame period. Time periods 1761, 1762 
are each a field period. Signal VSync goes active at the 
15 start of each field and remains active for time 1763 , 
i.e., for the vertical blanking lines, and then goes 
inactive at the start of the active data. For NTSC, 
signal VSync is active for lines 1 through 21 and line 262 
in field one, and for lines 263 through 284 and line 525 
20 in field two. Signal VSync remains inactive during time 
period 1764 of active data, e.g., 240 lines. Signal HSync 
is active during the horizontal blanking interval and 
inactive during the active portion of each line time 
including lines of the vertical blanking interval. 
25 Consequently, AND gate 1722, which has an input terminal 
that inverts signal HSYNC, generates an active signal that 
is negated by NOR gate 1723. The low signal from NOR gate 
1723 clears counter 1716. After two clock periods, the 
high signal from AND gate 1722 clears accumulator 1706. 
3 0 Signals VALID* and RESET* are provided by statistical 

analyzer timing controller 1650 (Figure 16), hereinafter 
controller 1650. Controller 1650 also generates a reset 
pulse on line FIELDRST* (Figure 17B) on each rising edge 
of signal VSync, i.e., at each field signal transition,' 
35 to reset the field memory address pointers. Similarly, 
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controller 1650 generates a pulse on line LINERST* at the 
rising edge of horizontal sync signal HSYNC to reset the 
line buffer address pointers. 

Figure 18 is a more detailed block diagram of one 
5 embodiment of reordering memory 403. In this embodiment, 
reordering memory 4 03 includes two first- in- first -out 
memories (FIFOs) 1801, 1802 that each receive 16 bits of 
data from field memory 451. The sixteen bits of data are 
pixel luminance and chrominance data. 

10 FIFOs 1801 , 1802 store alternate pixels from the 

active portion of each line. A FIFO controller (not 
shown) controls the alternation between FIFO 1801 and 
FIFO 18 02. The FIFO controller includes a state machine 
to control the writing to FIFOs and a pixel counter to 

15 track the location within the line. When the halfway 
point in the active portion of each line is reached, the 
FIFO controller generates a signal HALFLINE. In response 
to signal HALFLINE, a VRAM controller 1820, which is 
described more completely below, for reordering memory 4 03 

20 initiates transfer of data from FIFOs 1801, 1802 to video 
random access memory (VRAM) banks 1810, 1811. 

FIFO 18 01 drives a first bank of VRAM 1810 and. 
FIFO 1802 drives a second bank of VRAM 1811. Each VRAM 
bank 1810, 18il includes 5 pages of VRAM (Figure 19) , 

25 where a page is low- byte 256K memory chip and a high -byte 
256K memory chip. Each bank has ten 256Kx8 VRAMs . 

Data in VRAM banks 1810, 1811 is stored in raster 
order, but the field data is. spread across VRAM banks 
1810, 1811 to provide adequate memory bandwidth. Raster 

3 0 storage allows easy conversion from interlaced to frame 
ordering. Banks 1810, 1811 store alternate words. A word 
is an 8-bit luminance sample and an 8 -bit chrominance 
sample. Corresponding pages from the two banks store 
words from the same line, e.g., page one of bank one and 

35 page one of bank two receive data from the same video 
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lines. The horizontal blanking samples from each line do 
not appear in reordering memory 4 03, but samples from the 
active line time of vertical blanking interval 1764 do 
appear. (See Figure 17B.) This requires a maximum of 625 
5 lines per frame or 313 lines per field for PAL standard 
video . 

To make addressing simpler, in one embodiment, 320 
lines per field and 720 words per line are allocated in 
reordering memory 403. Consequently, for reordering 

10 memory 403 to hold eight fields requires 2560 lines (320 
lines/field * 8 fields) or five 256Kxl6 (10 256Kx8 VRAMs) 
memories per bank. The first sample resides in column 
zero of bank one; the second sample of each line, in 
column zero of bank two. The number of lines utilized in 

15 reordering memory 4 03 is determined by the number of 
active lines in a field. 

VRAM controller 1820 includes write control logic and 
read control logic that is shared by VRAM 

banks 1810, 1811. The write control logic writes sixteen 
20 bits, which is sometimes called a word, of video 

information, i.e., a pixel luminance sample and a pixel 
chrominance sample, to each VRAM bank 1810, 1811 
simultaneously. Controller 402 (Figures 4A and 4B) 
selects the starting address of the next field in 
25 reordering memory 403 that the write control logic circuit 
loads, and tracks the addresses of the fields residing in 
reordering memory 403. Similarly, as described above, 
controller 4 02 loads registers in the read control logic 
with the, addresses of the fields to be read from 
3 0 reordering memory 4 03. 

. VRAM controller 1820 runs on the 27 MHz video clock 
and reads from FIFOs 1801, 1802, writes to the DRAM port 
of VRAM banks 1810, 1811, performs DRAM refreshes, and 
transfers rows of data from VRAM banks 1810, 1811 to SAM 
35 port 1812, 1813. VRAM controller 1820 includes a state 
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machine to implement the column address strobes (CAS) 
before row address strobe (RAS) refreshes, 128-word page 
mode writes, and DRAM - to - SAM full line transfers. 

SAM ports 1812., 1813 from VRAM banks 1810, 1811 
5 (Figures 18 and 19) alternately supply data on output 
line 320 via multiplexer 1814. A 6.75 MHz serial clock 
clocks terminal SC (Figure 19) so that an output signal is 
provided from both VRAM banks 1810, 1811 every other 13.5 
MHz clock cycle. A serial port enable signal, that is 

10 active low., is applied to terminal SE for a single page of 
both banks and so allows only that page to drive the 
output bus from the VRAM bank to multiplexer 1814. 
Multiplexer 1814 selects the output signal from each bank 
1810, 1811 on alternate 13.5 MHZ clock cycles on line 

15 SELECT. 

Figure 20 is a schematic diagram of the write and 
read address generators in VRAM controller 1820. The 
write address generator includes write field address 
register 2001 which is loaded by controller 402. 

2 0 Similarly, read address generator includes first parity 
field read address register 2010 and second parity field 
address register 2020 which are also loaded by 
controller 402, as described above in the discussion of 
state machine 480. 

25 In field mode, i.e., an output format code of Fl, a 

new value is loaded in register 2010 at the start of each 
field. In frame mode, i.e., an output format code of F0 , 
a new- value is loaded in registers 2010 and 2020 at the 
start of each frame. For frame mode, both row 

30 counter 2011 and row counter 2021 supply memory addresses 
for alternate lines on bus ADDR. 

After a reset signal to VRAM controller 1820, a VRAM 
controller state machine looks for an active portion of 
the first field. Figure 21 is a state diagram of the VRAM 

35 controller state machine. The VRAM controller state 
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machine has twenty states SO to S19. However, the twenty 
states are used in seven discrete steps as illustrated in 
Figure 21. 

Initially in state SO, the VRAM controller state 
5 machine is in a first wait step 2110 after receipt of the 
reset signal. The control state machine remains in wait 
step so long as either vertical sync signal VSync is 
active or field signal FIELD is inactive. See Figure 4A. 

When field signal FIELD goes active and vertical sync 
10 signal VSync goes inactive, i.e, during of the active 
portion of the even parity field, the VRAM controller 
state machine transitions from state SO in first wait 
step 2110 to state SI in a second wait step 2120. A 
subsequent falling edge on field signal FIELD indicating 
15 the start of the following odd parity field causes VRAM 
controller state machine to transition to state S2 in step 
load row counters 213 0. 

In load row counters step 213 0, which is during 
horizontal blanking and preceding transfer cycle 
20 step 2140, VRAM controller 1820 drives signal WALDRC* to 
terminal LD of write address row counter 2 0 02 active so 
that the contents of write field address register 2 001 are 
loaded in write address row counter 2002. The least 
significant six bits of write address counter .2 002 are 
25 loaded with a value of zero because controller 402 

(Figure 4B) specifies only the 320-line field store and 
not the exact row address. Loading write row address 
counter 2.002 initializes the row count for a field memory 
within' reordering memory 4 03. 
30 m load row counter steps 2130, VRAM controller 1820 

also drives signal FOLdC* to terminal LD of first field 
row counter 2011 and signal FILdC* to terminal LD of 
second field row counter 2 021 active if the output format 
code is F0 and only terminal LD of odd row. counter 2011 if 
35 the output format code is Fl. The active signals on load 



■ V; 



i 

WO 95/15659 



i 



PCT/US94/13287 



- 123 - 

terminals LD loads the read addresses from registers 2010, 
2020 into row counters 2011, 2021 respectively. The least 
significant six bits of read address counters 2011, 2021 
are loaded with a value of zero because controller 4 02 
5 specifies only the 320 -line field store and not the exact 
row address. The high level of horizontal blanking 
signal HSYNC resets flag HALFLINE. Upon completion of 
horizontal blanking, i.e, a high-to-low transition of 
horizontal sync signal HSYNC that indicates the start of 

10 active video in the line, the VRAM controller state 
machine transitions from state S2 in load counters 
step 2130 to state S3 in transfer cycle step 2140. 
States S3 to S7 are within transfer cycle step 2140. At 
the high- to -low transition of horizontal sync signal 

15 HSYNC, the FIFO controller begins loading data in FIFOs 
1801, 1802 (Figure 18) . 

In transfer cycle step 2140, a DRAM- to -SAM transfer 
moves an entire line of data to the SAM port for serial 
access. The transfer cycle specifies column zero as the 

20 start tap for the SAM port. Upon transfer to the SAM 
port, the data is ready for reading during the active 
portion of the line. 

Figure 22 is a timing diagram for transfer cycle 
step 2140. The transfer cycle requires five clock cycles 

25 on the 27 MHz clock line which is labeled "27 MHz". On 
the rising edge of the first clock cycle, the VRAM 
controller state machine transitions from state S2 to 
state S3 as shown on line labeled SB (5:0). Also, the 
signal on line WALDRC* to load terminal LD of write row 

30 counter 2002 goes active to enable loading of -write row 
counter 2002. 

On the rising edge of the second clock cycle, the 
VRAM controller state machine transitions from state S3 to 
state S4 as shown on line labeled SB (5:0) . Also, the 

35 signal on line WALDRC* to load terminal LD of write row 
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counter 2002 goes inactive and the signal on transfer 
terminal TR*/0E* of each VRAM (Figure 19), the signal on 
line WACCLR* to clear terminal CLR of column counter 2003, 
and the signal on line F0AOE* of three-state buffer 2012 
5 go active. Hence, three-state buffer 2012 drives the 
address in odd row counter 2011 onto address bus ADDR and 
column counter 2003 is cleared. The signal on line WACOE* 
goes inactive so that three-state buffer 2005 has a high 
impedance . 

10 On the third rising clock edge in transfer" cycle 

step 2140 (Figure 22) , the VRAM controller state machine 
transitions from state S4 to state S5 as shown on line 
labeled SB (5:0) . Also, the signal on line WACCLR* to 
clear terminal CLR of column counter 2 003 goes inactive 
15 and the signal on line RAS* to VRAM banks 1810, 1811 goes 
active. The most significant bits WrVSel[2:0] of write 
row counter go to the VRAM controller state machine. The 
VRAM controller state machine uses these bits to determine 
the VRAMs currently receiving the write data. The VRAM 
20 controller state machine asserts the appropriate signal on 
the appropriate read address strobe RAS* to VRAM banks 
1810, 1811, but in Figure 22, the signal is shown 
generically as simply RAS*. 

On the fourth rising clock edge in transfer cycle 
25 step 2140, the VRAM controller state machine transitions 
from state S5 to state S6 as shown on line labeled 
SB (5:0). Also, the signal on transfer terminal TR*/OE* of 
each VRAM (Figure 19), and the signal on line F0AOE* of 
three-state buffer 2012 go inactive. The signal on line 
30 WACOE* goes active so that three-state buffer 2005 passes 
the address, i.e., TAP=0, from column counter 2003 to 
bus ADDR. 

On the fifth rising clock edge in transfer cycle 
step 214 0, the VRAM controller state machine transitions. 
3 5 from state S6 to state S7 as shown on line labeled 
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SB (5:0) . The signal on column address strobe line CAS* 
goes active to VRAM banks 1810, 1811 and the signal on 
line FOACEN* to enable terminal CE of row counter 2011 
goes active so that row counter 2011 is incremented. At 
5 the end of the fifth clock cycle, all active signals go 
inactive and transfer cycle step 2140 is complete. 

Upon completion of transfer cycle step 2140, 
processing transfers to refresh cycle step 2150 in which 
states S8 to S12 of the VRAM controller state machine 

10 refresh VRAM banks 1810, 1811. Figure 23 is a timing 
diagram for refresh cycle step 2140. As shown in 
Figure 23, the refresh cycle requires five 27 MHz clock 
cycles and the signals on lines RAS*, CAS* , and DSF to 
VRAM banks 1810, 1811 are driven active in the sequence 

15 shown; 

VRAM controller 1820 includes a refresh timer. Each 
time refresh cycle step 2140 is executed, the refresh 
timer is cleared. When the refresh timer times out, a 
REFRESH flag is set. Upon completion of refresh cycle 

20 step 2150, processing transfers to wait cycle step 2160. 
Wait cycle step 2160 is state S13 of the VRAM 
controller state machine. Recall that when horizontal 
sync signal HSYNC went inactive, the FIFO controller 
started loading FIFOs 1801, 1802. When FIFOs 1801, 1802 

25 are half full, FIFO controller generates an active signal 
on line HALFLINE, as explained above. 

The VRAM control state machine remains in state S13 
as long as both flags HALFLINE and REFRESH are inactive. 
If flag HALFLINE does not go active before flag REFRESH 

3 0 goes active, the VRAM controller state machine returns to 
refresh cycle step 215b upon flag REFRESH being set. The 
refresh time times out and sets flag REFRESH every two 
hundred fifty-six 27 MHz clock cycles. This schedule 
completes the 512 refreshes necessary to sustain 

35 reordering memory 403 in 4.85 msec (512*256/27*10 6 ) . 
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If flag HALFLINE is set and flag REFRESH is not set, 
i.e., the refresh timer has not yet timed out, the VRAM 
controller state machine transitions from wait cycle 
step 2160 to 128-word page mode write cycle 2170 that 
5 includes states S14 to S19. A write from FIFOs 1801, 1802 
to VRAM banks 1810, 1811 requires two hundred sixty-one 27 
MHz clock cycles, i.e., three clock cycles for row address 
strobe line RAS* precharge, two clock cycles per write, 
plus two additional clock cycles. One of the precharge 
10 cycles falls in wait state step 2160 of the VRAM 

controller state machine. Figure 24 is a timing diagram 
of page mode write cycle step 2170. 

Upon entry to page mode write cycle 2170, on the 
first rising clock edge (line 27MHz in Figures 18, 23), 
15 VRAM controller state machine transitions to state S14 as 
shown in line SB (5:0) . On the second rising clock edge in 
page mode write cycle 2170, the VRAM controller state 
machine transitions to state S15; the signal on write 
address column out enable line WACOE* goes inactive; and 
2 0 the signal on line write address row out enable line 

WAROE* goes active. Hence, three-state buffer 2004 passes 
the value in row counter 2002 to bus ADDR as shown on line 
ADDR of Figure 24 . 

On the third rising clock edge in page mode write 
25 cycle 2170, the VRAM controller state machine transitions 
to state S16, and the signal on line RAS* to VRAM banks 
1810, 1811 goes active. For simplicity in Figure 24 lines 
RAS1* to RAS5* are simply given as RAS*. As explained 
above; the most significant bits WrVSel[2:0] of write row 
30 counter 2002 go to the VRAM controller state machine and 
the VRAM controller state machine, based on those bits, 
drives the appropriate row address strobe line active. 

On the fourth rising clock edge in page mode write 
cycle 2170, the VRAM controller state machine transitions 
35 to state S17. The signals on write enable line WE* to 
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VRAM banks 1810, 1811 and on write address column out 
enable line WACOE* go active while the signal on line 
write address row out enable line WAROE* goes inactive. 
The signals on lines WE* , WACOE* and WAROE* do not change 
5 throughout the remainder of page mode write cycle 2170 . 
In response to the active signal on line WACOE * , 
three- state buffer 2005 passes value COLO in column 
counter 2003 to bus ADDR as shown on line ADDR of 
Figure 24. When signal FR* goes active, the signal on the 

10 read enable terminal of FIFO 1801, 1802 is active. The 
value read from the FIFO is stored in registers 183 0 
and 1831 respectively because signal DCKEN* drives a clock 
enable terminal of registers 1830 and 1831. Registers 
183 0 and 1831 store the data output from the respective 

15 FIFO and provide the data to the respective VRAM bank as 
illustrated in Figure 18. 

On the fifth rising clock edge in page mode write 
cycle 2170, VRAM controller state machine transitions to 
state S18. The signals on column address strobe line CAS* 

20 to VRAM banks 1810, 1811 and on write address column 
counter enable line WACCEN* to terminal CE of column 
counter 2003 are driven active. Thus, column counter 2003 
is incremented on the next rising edge of 27 MHz clock and 
the address on bus ADDR is incremented. 

25 As shown in Figure 24 , page mode write cycle 

continues to cycle between states S17 and SI 8 of the VRAM 
controller state machine until 12 8 words have been written 
from FIFOs 1801, 1802 to VRAM banks 1810, 1811. Upon 
completion of the write to VRAM banks 1810, 1811, the VRAM 

3 0 controller state machine goes to state S19 . 

In state S19, signal WATCEN* to terminal CE of write 
row counter 2002 is driven active and so increments row 
counter 2002. Hence, row counter 2002 is incremented at 
the end of each line. Signal WATCClr* is also driven 
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active and so resets write column counter 2003 at this 
time . 

Upon completion of the operations in state S19, the 
VRAM controller state machine transfers to another step 
5 depending on the state of horizontal sync signal HSYNC and 
new field flag NEW FIELD. New field flag NEW FIELD is 
valid only when horizonal sync signal HSYNC is active. 
When horizonal sync signal HSYNC is active, it means that 
a new line is being transferred into FIFOs 1801, 1802. 

10 if horizontal sync signal HSYNC and new field flag 

NEW FIELD are both active, processing transfers from page 
mode write step 2170 to load row counters step 2130. If 
horizontal sync signal HSYNC is active and new field 
signal NEW FIELD is inactive, processing transfers from 

15 page mode write step 2170 to transfer cycle step 2140. If 
horizontal sync signal HSYNC is inactive, processing 
transfers from page mode write step 2170 to refresh cycle 
step 2160. 

After completion of the first page mode write 
20 cycle 2170 after flag HALFLINE is set, horizontal sync 
signal HSYNC is not active and so the VRAM controller 
state machine branches to refresh cycle 2150, and performs 
a refresh cycle. Similarly, after completion of the 
second page mode write cycle, horizontal sync signal HSYNC 
25 is not active and so VRAM controller state machine 
initiates yet another refresh cycle 2150. 

Writing 720 bytes to VRAM banks 1810, 1811 from FIFOs 
18.01, 1802 requires seven hundred ninety- three 27 MHz 
cycles ((720 bytes / (256 bytes per write cycle)) * (261 
30 clock cycles/write cycle) + 5 clock cycles for first 

refresh cycle + 5 clock cycles for second refresh cycle) . 
Note that only 720 bytes are considered because on each 
write a byte from FIFO 1801 and a byte from FIFO 1802 are 
simultaneously written to VRAM banks 1810, 1811. The data 
5 transfer from FIFOs 1801, 1802 to VRAM banks 1810,. 1811 
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starts after 3 60 pixels have been loaded into FIFOs 1801, 
1802. One 27 MHz clock cycle is required to write a byte 
to FIFOs 1801, 1802. Thus, to write the last 360 pixels 
to FIFOs requires seven hundred twenty 27 MHz clock. 
5 cycles. Hence, the transfer of a line of pixel data from 
FIFOs 1801, 1802 to VRAM banks 1810, 1811 is complete 73 
(793-720) clock cycles into the horizontal blanking for 
the next line. 

The line of data loaded into the SAM port during 

10 transfer cycle step 213 0 is moved onto serial output 

line 3 02 under the control of a SAM port state machine. A 
timing diagram for the SAM port state machine that shows 
the various states is given in Figure 25. As long as 
horizontal sync signal HSYNC is active, the SAM port state 

15 machine is in a wait state SI. When horizontal sync 

signal HSYNC falls, the SAM port state machine transitions 
to state S2 upon a rising edge of the 27 MHz clock. In 
state S2 , the SAM port state machine waits five 13 . 5 MHz 
clock cycles for transfer cycle step 213 0 to complete and 

2 0 then starts to alternate between states S3 and S4 to move 

data onto output line 3 02, which is shown as line MUXOUT 
in Figure 25. Line PXCNT is a counter within the timing 
controller for the SAM port that tracks the position 
within each video line. 
25 The most significant bits R0VSel[2:0] of read row 

counter 2011 and the most significant bits RlVSel[2:0] of 
read row counter 2 021 are provided to the SAM port state 
machine. Based on the values of these bits the SAM port 
state machine determines which of the five serial data out 

3 0 enable signals SE1* to SE5* signals is low for each line 

time.- In field mode, the address bits from row 
counter 2011 determine which serial data out enable signal 
SE* signal is low. For frame mode operation, two fields 
alternately supply lines of output data. Therefore, row 
3 5 counter 2 011 provides the address bits used to select the 
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serial data out enable signal SE* during the even numbered 
lines and row counter 2 021 provides the address bits used 
to select the serial data out enable signal SE* during the 
odd numbered lines ■ (assuming line numbers begin with 0) . 
5 In Figure 25, the serial clock line SC* and the serial 
data out enable line SE* are shown as SCI* and SE1* 
respectively. The low signal on line SE1* enables the 
output buffers for one set of VRAMs from each bank 1810 
and 1811. 

10 The signal on line SCI provides 360 rising edges per 

line time. This sequence reads 720 samples after 720 13.5 
MHz clock cycles from tap addresses zero to 359 of both 
banks. Once the 720 samples are read out, the cycles on 
line SCI stop, and the SAM port state machine waits for 
15 the active. portion of the next line. During blanking, 
pullups on the data busses provide OxFF input signals to 
both ports of multiplexer. 

The embodiments described above of the video data 
stream analyzer and the methods for analyzing a video data 
20 stream are illustrative only and are not intended to limit 
the invention to the particular embodiments described. In 
view of this disclosure, those skilled in the area can 
implement the principles of this invention in a wide 
variety of architectures and in a wide variety of 
25 applications. For example, the thresholds used in the 
above methods have been defined in terms of the number of 
pixels in a line and the number of pixels in a field. 
Therefore, the principles of this invention are not 
limited to a particular frame size but can be readily 
30 applied to a frame of any size. Moreover, the methods of 
this invention can be utilized in non-real time 
applications as well as real time applications. For 
example, a video data stream could be analyzed in non-real 
time according to the principles of this invention and the 
3 5 results stored for subsequent use by an encoding system or 
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by a broadcast system. The principles of this invention 
are generally applicable to any application that requires 
determination of the characteristics of the data in a 
video data stream. • Also, while in the embodiments 

5 described above, a repeated field is dropped, in another 
embodiment, a repeated field might be averaged with the 
duplicate field, i.e., corresponding pixels could be 
summed and the result divided in half. This could reduce 
any noise present in the original repeated field. Thus, 

0 the principles of the invention can indeed be implemented 
in a wide variety of ways . 
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APPENDIX I 



The operation -of state machine 480 in combination 
with statistical processor 490, that performs the scan 
pattern detection, and repeated field detection using the 
5 primary and secondary criteria, attempts to recognize and 
repair 72 different input video data streams to video data 
stream analyzer 300. The following tables give an example 
of each of the sequences. The input video data streams 
given below are not inclusive of all input video data 
10 streams processed by video data stream analyzer 300 and so 
are illustrative only. The examples given below are not 
intended to limit the invention to the particular input 
video data streams given. 

In the tables, the following conventions are 
15 employed: 

Frames a, b, c, d, e, belong to one 

sequence . 

Frames u, v, w, x, y, belong to a 

different sequence. 
20 "In:" depicts input video data stream 301 

to video data stream analyzer 300. 
"Out:" depicts output video sequence 302 of 
video data stream analyzer 3 00 and 
hence is the signal that is eventually 
25 encoded and transmitted. 

' v "Display code:" depicts, the display code 

.,, .associated with each, transmitted frame 
. as defined in Table 3 . 
"Display:" depicts what a decoding system should 
display at 30 fps given the output signal 
of the video data stream analyzer and the 
display code. Note that the ultimately 
displayed sequence may contain noise and/or 
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distortion introduced by any subsequent 
operation on the output data stream from 
the video data stream analyzer, e.g., if 
the sequence is passed through a lossy 
5 encoder-decoder before being displayed. 



1 . Pure Video 

In: ala2 blb2 clc2 dld2 ele2 ... 

Out: ala2 blb2 clc2 dld2 ele2 ... 
Display 

10 code: 02 02 02 02 02... 
Dis- 
play: ala2 blb2 clc2 dld2 ele2 . . . 

2. Video (mid- frame edit): 

In: ala2 blb2 clu2 vlv2 wlw2 ... 

15 Out: ala2 blb2 u2u2 vlv2 wlw2 ... 
Display 

code: 02 02 02 02 02 
Dis- 
play: ala2 blb2 u2u2 vlv2 wlw2 . . . 

2 0 3. Video to movie 1: 

In: ulu2 vlv2 ala2 alb2 blc2 clc2 dld2 ala2 . . . 

Out: ulu2 vlv2. ala2 blb2 clc2 dld2 ala2 

Display 

code: 02 02 03 12 13 02 03 . 
25 Dis- 
play: ulu2 vlv2 ala2 alb2 bic2 clc2 dld2 ala2 

4 . Video to movie 2 : 

In: ulu2 vld2 ala2 alb2 blc2 clc2 dld2 ala2 ... 
Out: ulu2 d2d2 ala2 blb2 clc2 dld2 ala2 ... 

3 0 Display 
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code: 02 02 03 12 13 02 03 

Dis- 
play: ulu2 d2d2 ala2 alb2 blc2 clc2. dld2 ala2 

5 . Video to movie 3 : 

5 In: ulu2 vlc2 dld2 ala2 alb2 blc2 clc2 dld2 ... 
Out: ulu2 c2c2 dld2 ala2 blb2 clc2 dld2 

Display 

code: 02 02 02 03 12 13 02 

Dis- 

10 play: ulu2 c2c2 dld2 ala2 alb2 blc2 clc2 dld2 

6 . Video to movie 4 : 

In: ulu2 vlc2 clc2 dld2 ala2 alb2 blc2 clc2 
Out: ulu2 c2c2 clc2 dld2 ala2 blb2 clc2 

Display 

15 code: 02 02 02 02 03 12 13 

Dis- 
play: ulu2 c2c2 clc2 dld2 ala2 alb2 blc2 clc2 

7 . Video to movie 5 : 

In: ulu2 vlb2 blc2 clc2 dld2 ala2 alb2 blc2 
20 Out: ulu2 b2b2 blbl clc2 dld2 ala2 blb2 clc2 

Display 

code: 02 02 02 02 02 03 12 13 

Dis- 
play: ulu2 b2b2 blbl clc2 dld2 ala2 alb2 blc2 

25 8. Video to movie 6: 

In: ulu2 vla2 alb2 blc2 clc2 dld2 ala2 alb2 
Out: ulu2 a2a2 alb2 blbl clc2 dld2 ala2 blb2 

Display 

code: 02 02 02 02 02 02 03 12 

30 Dis- 
play:. ulu2 a2a2 alb2 blbl clc2 dld2 ala2 alb2 



WO 95/15659 



FOYUS94/ 13287 



9 . Video to movie 7 : 

In: ulu2 vlv2 blc2 clc2 
Out: ulu2 vlv2 blbl 
Display 
5 code: 02 02 02 
Dis- 
play: ulu2 vlv2 blbl 

10. Video to movie 8: 
In: ulu2 vlv2 alb2 blc2 

10 Out: ulu2 vlv2 alb2 

Display 

code: 02 02 02 

Dis- 
play: ulu2 vlv2 alb2 

15 11. Video to movie 9: 

In: ulu2 vlv2 clc2 dld2 

Out: ulu2 vlv2 clc2 

Display 

code : 02 02 02 

20 Dis- 
play: ulu2 vlv2 clc2 

12 . Video to movie 10 : 
In: ulu2 vlv2 dld2 ala2 
Out: ulu2 vlv2 dld2 

25 Display 

code: 02 02 02 

Dis- 
play: ulu2 vlv2 dld2 

13. Movie to video 1: 

30 In: ala2 alb2 blc2 clc2 
Out: ala2 blb2 
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dld2 ala2 alb2 blc2 . . . 
clc2 dld2 ala2 blb2 clc2 

02 02 03 12 13 . . . 

clc2 dld2 ala2 alb2 blc2 

clc2 dld2 ala2 alb2 . . . 
blbl clc2 dld2 ala2 blb2 

02 02 02 03 12 

blbl clc2 dld2 ala2 alb2 . . . 

ala2 alb2 blc2 clc2 . . . 
dld2 ala2 blb2 clc2 . . . 

02 03 12 13 

dld2 ala2 alb2 blc2 clc2 

alb2 blc2 clc2 dld2 
ala2 blb2 clc2 dld2 

03 12 13 02 . . . 
ala2 alb2 blc2 clc2 dld2 . ... 

dld2 ulu2 vlv2 wlw2 . . . 
clc2 dld2 ulu2 vlv2 wlw2 . . . 
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Display 

code: 03 12 13 02 02 02 02 

Dis- 
play: ala2 alb2blc2 clc2 dld2 ulu2 vlv2 wlw2 

5 14 . Movie to video 2 : 

In: ala2 alb2 blc2 clc2 ulu2 vlv2 wlw2 xlx2 
Out: ala2 blb2 C lc2 ulu2 vlv2 wlw2 xlx2 

Display 

code: 03 02 13 02 02 02 02 

10 Dis- 
play: ala2 alb2 blc2 clc2 ulu2 vlv2 wlw2 xlx2 

15 . Movie to video 3 : 

In : dld2 ala2 alb2 blc2 ulu2 vlv2 wlw2 xlx2 
Out: dld2 ala2 blb2 ulu2 vlv2 wlw2 xlx2 

15 Display 

code: 02 03 13 02 02 02 02 

Dis- 
play: dld2 ala2 alb2 blb2 ulu2 vlv2 wlw2 xlx2 

16 . Movie to video 4 : 

20 In: clc2 dld2 ala2 alb2 ulu2 vlv2 wlw2 xlx2 

Out: clc2 dld2 ala2 ulu2 vlv2 wlw2 xlx2 

Display 

code: 02 02 04 02 02 02 02 

Dis- 

25 Play: C lc2 dld2 ala2 ala2 ulu2 vlv2 wlw2 xlx2 

17. Movie to video 5: 

In: clc2 dld2 ala2 ulu2 vlv2 wlw2 xlx2 yly2 

Out: clc2 dld2 ala2 ulu2 vlv2 wlw2 xlx2 yly2 

Display 

30 code: 02 02 02 02 02 02 02 02 

Dis- 
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play: clc2 dld2 ala2 ulu2 vlv2 wlw2 xlx2 yly2 ... 

18 . Movie to video 6 : 

In: clc2 dld2 alu2 vlv2 wiw2 xlx2 yly2 . . . 
Out : clc2 dld2 u2u2 vlv2 wlw2 xlx2 yly2 . . . 

5 Display 

code: 02 02 02 02 02 02 02 . . . 

Display: clc2 dld2 u2u2 vlv2 wlw2 xlx2 yly2 . . . 

19 . Movie to video 7 : 

In: clc2 dld2 ala2 alu2 vlv2 wlw2 xlx2 yly2 . . . 
10 Out: clc2 dld2 ala2 vlv2 wlw2 xlx2 yly2 ... 

Display 

code: 02 02 04 02 02 02 02 

Dis- 
play: clc2 dld2 ala2 ala2 vlv2 wlw2 xlx2 yly2 . . . 

15 20. Movie to video 8: 

In: clc2 dld2 ala2 alb2 blu2 vlv2 wlw2 xlx2 ... 

Out : clc2 dld2 ala2 blb2 vlv2 wlw2 xlx2 . . . 

Display 

code: 02 02 03 13 02 02 02 
20 Dis- 
play: clc2 dld2 ala2 alb2 blb2 vlv2 wlw2 xlx2 

21. Movie to video 9: 

In: clc2 dld2 ala2 alb2 blc2 clu2 vlv2 wlw2 . . . 

Out : clc2 dld2 ala2 blb2 clc2 vlv2 wlw2 

25 Display 

code: 02 02 03 12 13 02 02 

Dis- . 

play: clc2 dld2 ala2 alb2 blc2 clc2 vlv2 wlw2 



22 . Movie to video 10 : 
3 0 In: dld2 ala2 alb2 blc2 clc2 dlu2 vlv2 wlw2 . 
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Out: dld2 ala2 blb2 clc2 u2u2 vlv2 wlw2 

Display 

code: 02 03 12 13 02 02 02 

Dis- 

5 play: dld2 ala2 alb2 blc2 clc2 u2u2 vlv2 wlw2 ... 



23. Movie to movie 1: (same as # 78) 

In: ala2 alb2 blc2 clc2 dld2 ulu2 ulv2 vlw2 

Out: ala2 blb2 clc2 dld2 ulu2 vlv2 

Display 

10 code: 03 12 13 02 03 12 
Dis- 
play: ala2 alb2 blc2 clc2 dld2 ulu2 ulv2 vlw2 

24. Movie to movie 2: (same as # 73) 

In: ala2 alb2 blc2 clc2 ulu2 ulv2 vlw2 wlw2 
15 Out: ala2 blb2 clc2 ulu2 vlv2 wlw2 

Display 

code: 03 12 13 03 12 13 . . 

Dis- 
play: ala2 alb2 blc2 clc2 ulu2 ulv2 vlw2 wlw2 

20 25. Movie to movie 3: (same as # 68) 

In: ala2 alb2 blc2 ulu2 ulv2 vlw2 wlw2 xlx2 

Out : ala2 blb2 ulu2 vlv2 wlw2 xlx2 

Display 

. code: 03 13 03 12 13 02 
25 Dis- 
play: ala2 alb2 blb2 ulu2 ulv2 vlw2 wlw2 xlx2 

26. Movie to movie 4: (same as # 63) 

In: ala2 alb2 ulu2 ulv2 vlw2 wlw2 xlx2 ulu2 
Out: ala2 ulu2 vlv2 wlw2 xlx2 ulu2 

3 0 Display 
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code: 04 03 12 13 02 03 . . . 

Display: ala2 ala2 ulu2 ulv2 vlw2 wlw2 xlx2 ulu2 . . . 

27. Movie to movie 5: (same as # 58) 

In: clc2 dld2 ala2 ulu2 ulv2 vlw2 wlw2 xlx2 . . . 

. 5 Out : clc2 dld2 ala2 ulu2 vlv2 wlw2 xlx2 . . . 

Display 

code: 02 02 02 03 12 13 02 
Dis- 
play: clc2 dld2 ala2 ulu2 ulv2 vlw2 wlw2 xlx2 . . . 

10 28. Movie to movie 6: 

In: dld2 alu2 ulv2 vlw2 wlw2 xlx2 ulu2 ulv2 ... 

Out : dld2 u2u2 ulv2 vlvl wlw2 xlx2 ulu2 vlv2 ... 

Display 

code: 02 02 02 02 02 02 03 12 . . . 
15 Dis- 
play: dld2 u2u2 ulv2 vlvl wlw2 xlx2 ulu2 ulv2 . . . 

29. Movie to movie 7: 

In: dld2 ala2 alu2 ulv2 vlw2 wlw2 xlx2 ulu2 . . . 

Out : dld2 ala2 ulv2 vlvl wlw2 xlx2 ulu2 

20 Display 

code: 02 04 02 02 02 02 03 ... 
Dis- 
play: dld2 ala2 ala2 ulv2 vlvl wlw2 xlx2 ulu2 

30. Movie to movie 8: 

25 In: dld2 ala2 alb2 blu2 ulv2 vlw2 wlw2 xlx2 ... 

Out : dld2 ala2 blb2 ulv2 vlvl wlw2 xlx2 . . . 

Display 

code: 02 03 13 02 02 02 02 

Dis- 

3 0 play: dld2 ala2 alb2 blb2 ulv2 vlvl wlw2 xlx2 . . . 
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31. Movie to movie 9: 
In: ala2 alb2 blc2 clu2 
Out: ala2 blb2 clc2 
Display 

5 code: 03 12 13 
Dis- 
play: ala2 alb2 blc2 

32. Movie to movie 10 : 
In: ala2 alb2 blc2 clc2 

10 Out: ala2 blb2 

Display 

code: 03 12 

Dis- 
play: ala2 alb2 blc2 

15 33. Movie to movie 11: 

In: ala2 alb2 blc2 clc2 

Out: a ia2 blb2 
Display 

code: 03 12 

20 Dis- 
play: a ia2 alb2 blc2 

34. Movie to movie 12: 
In: blc2 clc2 dlw2 xlx2 
put :. • clc2 w2w2 

25 Display 

code: .13 02 
Dis- 
play: blc2 clc2 w2w2 

35. Movie to movie 13: 
3 0 In: ' blc2 clc2 dlw2 wlw2 
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ulv2 vlw2 wlw2 xlx2 . . . 

ulv2 vlvl wlw2 xlx2 . . . 

02 02 02 02 

clc2 ulv2 vlvl wlw2 xlx2 

dlu2 ulv2 vlw2 wlw2 . . . 
clc2 u2u2 ulv2 vlvl wlw2 . . . 

13 02 02 02 02 

clc2 u2u2 ulv2 vlvl wlw2 . . . 

dlx2 ulu2 ulv2 vlw2 ... 
clc2 x2x2 ulu2 vlv2 . . . 

13 02 03 12 

clc2 x2x2 ulu2 ulv2 vlw2 . . . 

ulu2 ulv2 vlw2 wlw2 ... 
xlx2 ulu2 vlv2 wlw2 

02 03 12 13 . . . 

xlx2 ulu2 ulv2 vlw2 wlw2 . . . 

xlx2 ulu2 ulv2 vlw2 . . . 
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Out : clc2 w2w2 wlw2 xlx2 ulu2 vlv2 ..... 

Display- 
code: 13 02 02 02 03 12 
Dis- 

5 play: blc2 clc2 w2w2 wlw2 xlx2 ulu2 ulv2 vlw2 ... 

36. Movie to movie 14: 

In: blc2 clc2 dlv2 vlw2 wlw2 xlx2 ulu2 ulv2 . . . 

Out: clc2 v2v2 w2w2 wlw2 xlx2 ulu2 vlv2 . . . 

Display 

10 code: 13 02 02 02 02 03 12 . . . 
Dis- 
play: blc2 clc2 v2v2 w2w2 wlw2 xlx2 ulu2 ulv2 . . . 

37. Movie to movie 15: 

In: ala2 alb2 blc2 clx2 ulu2 ulv2 vlw2 wlw2 ... 

15 Out : ala2 blb2 clc2 ulu2 vlv2 wlw2 

Display 

code: 03 12 13 03 12 13 
Dis- 
play: ala2 alb2 blc2 clc2 ulu2 ulv2 vlw2 wlw2 . . . 

20 38. Movie to movie 16: 

In: ala2 alb2 blc2 clw2 xlx2 ulu2 ulv2 vlw2 ... 

Out: ala2 blb2 clc2 xlx2 ulu2 vlv2 ... 

Display 

code: 03 12 13 02 03 12 
25 Dis- 
play: ala2 alb2. blc2 clc2 xlx2 ulu2 ulv2 vlw2 . . . 

39. Movie to movie 17: 

In: ala2 alb2 blc2 clw2 wlw2 xlx2 ulu2 ulv2 . . . 
Out: ala2 blb2 clc2 wlw2 xlx2 ulu2 vlv2 ... 
3 0 Display- 
code: 03 12 13 02 02 03 12 
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Dis- 
play: ala2 alb2 blc2 clc2 ' wlw2 xlx2 ulu2 ulv2 ... 

40. Movie to movie 18: 

In: alb2 blc2 clv2 vlw2 wlw2 xlx2 ulu2 ulv2 ... 
5 Out : blb2 clc2 vlvl wlw2 xlx2 ulu2 vlv2 . . . 
Display- 
code: 12 13 02 02 02 03 12 . . . 
Dis- 
play: alb2 blc2 clc2 vlvl wlw2 xlx2 ulu2 ulv2 ... 

.10 41. Movie to movie 19: 

In : alb2 blc2 clu2 ulv2 vlw2 wlw2 xlx2 ulu2 . . . 

Out : blb2 clc2 ulv2 vlvl wlw2 xlx2 ulu2 . . . 

Display 

code: 12 13 02 02 02 02 03 ... 

15 Dis- 
play: alb2 blc2 clc2 ulv2 vlvl wlw2 xlx2 ulu2 . . . 

42. Movie to movie 20: 

In: ala2 alb2 blx2 ulu2 ulv2 vlw2 wlw2 xlx2 ... 

Out : ala2 blb2 ulu2 vlv2 wlw2 xlx2 . . . 

2 0 Display 

code: 03 13 03 12 13 02 . . . 

Dis- 
play: ala2 alb2 blb2 ulu2 ulv2- vlw2 wlw2 xlx2 .... 

43. Movie to movie 21: 

25 -In: ala2 alb2 blw2 xlx2 ulu2 ulv2 vlw2 wlw2 ... 

Out: ala2 blb2 xlx2 ulu2 vlv2 wlw2 ... 
Display 

code: 03 13 02 03 12 13 ... 
Dis- 

3 0 play: ala2 alb2 blb2 xlx2 ulu2 ulv2 vlw2 wlw2 . . . 
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44. Movie to movie 22: 

In: ala2 alb2 blw2 wlw2 xlx2 ulu2 ulv2 vlw2 . . . 
Out : ala2 blb2 wlw2 xlx2 ulu2 vlv2 

Display 

5 code: 03 13 02 02 03 12 

Dis- 
play: ala2 alb2 blb2 wlw2 xlx2 ulu2 ulv2 vlw2 . . . 

45. Movie to movie 23: 

In: ala2 alb2 blv2 vlw2 wlw2 xlx2 ulu2 ulv2 ... 
10 Out: ala2 blb2 vlvl wlw2 xlx2 ulu2 vlv2 ... 

Display 

code: 03 13 02 02 02 03 12 

Dis- 
play : ala2 alb2 blb2 vlvl wlw2 xlx2 ulu2 ulv2 . . . 

15 46. Movie to movie 24: 

In: ala2 alx2 ulu2 ulv2 vlw2 wlw2 xlx2 ulu2 . . . 

Out : ala2 ulu2 vlv2 wlw2 xlx2 ulu2 . . . 

Display 

code: 04 03 12 13 02 03 
20 Dis- 
play: ala2 ala2 ulu2 ulv2 vlw2 wlw2 xlx2 ulu2 ... 

47. Movie to movie 25: 

In: ala2 alw2 Xlx2 ulu2 ulv2 vlw2 wlw2 xlx2 . . . 
Out : ala2 xlx2 ulu2 vlv2 wlw2 xlx2 ... 

25 Display 

code: 04 02 03 12 13 02 

Dis- 
play: ala2 ala2 xlx2 ulu2 uiv2 vlw2 wlw2 xlx2 ... 



4 8. Movie to movie 26: 
30 In: ala2 alw2 wlw2 xlx2 ulu2 ulv2 vlw2 wlw2 . . . 
Out: ala2 wlw2 xlx2 ulu2 vlv2 wlw2 
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Display- 
code; 04 02 02 ■ 03 12 13 
Dis- 
play: ala2 ala2 wlw2 xlx2 ulu2 ulv2 vlw2 wlw2 

5 49. Movie to movie 27: 

In: ala2 alv2 vlw2 wlw2 xlx2 ulu2 ulv2 vlw2 
Out: ala2 vlv2 wlw2 xlx2 ulu2 vlv2 

Display 

code: 03 12 13 02 03 12 

10 Dis- 
play: ala2 alv2 vlw2 wlw2 xlx2 ulu2 ulv2 vlw2 

50. Movie to movie 28: 

In: dld2 alx2 ulu2 ulv2 vlw2 wlw2 xlx2 ulu2 
Out : dld2 x2x2 ulu2 vlv2 wlw2 xlx2 ulu2 

15 Display 

code: 02 02 03 12 13 02 03 

Dis- 
play: dld2 x2x2 ulu2 ulv2 vlw2 wlw2 xlx2 ulu2 

51. Movie to movie 29: 
2 0 In: dld2 alw2 xlx2 ulu2 ulv2 vlw2 wlw2 xlx2 

Out: dld2 w2w2 xlx2 ulu2 vlv2 wlw2 xlx2 

Display 

code: 02 02 02 03 12 13 0 2 

Dis- 

25 r play,: dld2 w2w2 xlx2 ulu2 ulv2 vlw2 wlw2 xlx2 . 

52. Movie to movie 30: 

In: dld2 alw2 wlw2 xlx2 ulu2 ulv2 vlw2 wlw2 

Out: dld2 w2w2 wlw2 xlx2 ulu2 vlv2 wlw2 
Display 

30 code: 02 02 02 02 03 12 13 . 

Dis- 
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play: dld2 w2w2 wlw2 xlx2 ulu2 ulv2 vlw2 wlw2 ... 

53. Movie to movie 31: 

In: dld2 alv2 vlw2 wlw2 xlx2 ulu2 ulv2 vlw2 . . . 
Out : dld2 v2v2 vlvl wlw2 xlx2 ulu2 vlv2 

5 Display- 
code: 02 02 02 02 02 03 12 
Dis- 
play: dld2 v2v2 vlvl wlw2 xlx2 ulu2 ulv2 vlw2 . . . 

54. Movie to movie 32: 

10 In: dld2 ala2 xlx2 ulu2 ulv2 vlw2 wlw2 xlx2 . . . 
Out : dld2 ala2 xlx2 ulu2 vlv2 wlw2 xlx2 

Display 

code: 02 02 02 03 12 13 02 

Dis- 

15 play: dld2 ala2 xlx2 ulu2 ulv2 vlw2 wlw2 xlx2 . . . 

55. Movie to movie 33: 

In: dld2 ala2 wlw2 xlx2 ulu2 ulv2 vlw2 wlw2 . . . 
Out : dld2 ala2 wlw2 xlx2 ulu2 vlv2 wlw2 

Display 

20 code: 02 02 02 02 03 12 13 

Dis- 
play: dld2 ala2 wlw2 xlx2 ulu2 ulv2 vlw2 wlw2 ... 

56. Movie to movie 34: 

In : dld2 ala2 vlw2 wlw2 xlx2 ulu2 ulv2 vlw2 . . . 

25 Out: dld2 ala2 vlvl wlw2 xlx2 ulu2 vlv2 

Display 

code: 02 02 02 02 02 03 13, 
Dis- 
play: dld2 ala2 vlvl wlw2 xlx2 ulu2 ulv2 vlw2 ... 

0 57. Movie to movie 35: 
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In; dld2 ala2 ulv2 vlw2 wlw2 xlx2 ulu2 ulv2 
Out : dld2 ala2 ulv2 vlvl wlw2 xlx2 ulu2 vlv2 

Display- 
code: 02 02 02 02 02 02 03 12 
5 Dis- 
play: dld2 ala2 ulv2 vlvl wlw2 xlx2 ulu2 ulv2 

58. Movie to movie 36: 

In: dld2 ala2 ulu2 ulv2 vlw2 wlw2 xlx2 ulu2 

Out : dld2 ala2 ulu2 vlv2 wlw2 xlx2 ulu2 

10 Display 

code: 02 02 03 12 13 02 02 
Dis- 
play: dld2 ala2 ulu2 ulv2 vlw2 wlw2 xlx2 ulu2 

59. Movie to movie 37: 

15 In: ala2 alb2 xlx2 ulu2 ulv2 vlw2 wlw2 xlx2 

Out : ala2 xlx2 ulu2 vlv2 W lw2 xlx2 

Display 

code: 04 02 03 12 13 02 ... 

Dis- 

2 0 play: a ia2 ala2 xlx2 ulu2 ulv2 vlw2 wlw2 xlx2 

60. Movie to movie 38: 

In: ala2 alb2 wlw2 xlx2 ulu2 ulv2 vlw2 wlw2 

Out: ala2 wlw2 xlx2 ulu2 vlv2 wlw2 

Display 

25 code: 04 02 02 030 12 13 

Dis- \ . ' 

Play: ala2 ala2 wlw2 xlx2 ulu2 ulv2 vlw2 wlw2 

61. Movie to movie 39: 

In: ala2 alb2 vlw2 wlw2 xlx2 ulu2 ulv2 vlw2 
3 0 Out: a i a2 vlvl xlx2 ulu2 vlv2 

Display 
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code: 04 04 02 03 12 ... 

Dis- 
play: ala2 ala2 vlvl vlvl xlx2 ulu2 ulv2 vlw2 . . . 

62. Movie to movie 40: 

5 In: ala2 alb2 ulv2 vlw2 wlw2 xlx2 ulu2 ulv2 ... 
Out : ala2 ulv2 vlvl wlw2 xlx2 ulu2 vlv2 
Display- 
code: 04 02 02 02 02 03 12 ... 
Dis- 

10 play: ala2 ala2 ulv2 vlvl wlw2 xlx2 ulu2 ulv2 ... 

63. Movie to movie 41: 

In: ala2 alb2 ulu2 ulv2 vlw2 wlw2 xlx2 ulu2 . . . 

Out : ala2 ulu2 vlv2 wlw2 xlx2 ulu2 . . . 

Display 

15 code: 04 03 12 13 02 03 ... 
Dis- 
play: ala2 ala2 ulu2 ulv2 vlw2 wlw2 xlx2 ulu2 . . . 

64 . Movie to movie 42 : 

In: ala2 alb2 blc2 xlx2 ulu2 ulv2 vlw2 wlw2 . . . 
20 Out: ala2 blb2 xlx2 ulu2 vlv2 wlw2 ... 

Display 

code: 03 13 02 03 12 13 

Dis- 
play: ala2 alb2 blb2 xlx2 Ulu2 ulv2 vlw2 wlw2 ... 

25 65. Movie to movie 43: 

In: ala2 alb2 blc2 wlw2 xlx2 ulu2 ulv2 vlw2 . . . 
Out : - ala2 blb2 wlw2 xlx2 ulu2 vlv2 

Display 

code: 03 13 02 02 03 12 ... 

30 Dis- 
play: ala2 alb2 blb2 wlw2 xlx2 ulu2 ulv2 vlw2 . . . 
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66. Movie to movie 44 : 

In: ala2 alb2 blc2 vlw2 wlw2 xlx2 ulu2 ulv2 
Out: a ia2 blb2 vl vl wlw2 xlx2 ulu2 vlv2 

Display 



5 code: 03 13 

Dis- 
play: ala2 alb2 blb2 vlvl wlw2 xlx2 ulu2 ulv2 



02 02 02 03 12 
Dis- 



67. Movie to movie 45: 

In: a ia2 alb2 blc2 ulv2 vlw2 wlw2 xlx2 ulu2 
10 Out: ala2 blb2 ulv2 vlvl wiw2 

Display 

code: 03 13 02 02 02 02 03 

Dis- 
play: a ia2 alb2 blb2 ulv2 vlvl wlw2 xlx2 ulu2 

15 68. Movie to movie 46: 

In: ala2 alb2 blc2 ulu2 ulv2 vlw2 wlw2 xlx2 
Out: ai a2 blb2 U lu2 vlv2 wlw2 xlx 2 

Display 

code: 03 13 03 12 13 02 

20 Dis- 
play: ala2 alb2 blb2 ulu2 U1V2 vlw2 wlw2 xlx2 

69. Movie to movie 47: 

In: aia2 alb2 bic2 clc2 xlx2 uiu2 uiv2 vlw2 
.Out:. ala 2 blb2 clc2 xlx2 ulu2 ulv2 ' 

25 Display 

code': 03. 12 13 02 03 12 

Dis- 
play: ala2 alb2 blc2 clc2 xlx2 ulu2 ulv2 vlw2 

70. Movie to movie 48: 
30 in: ala2 alb2 blc2 clc2 W lw2 xlx2 ulu2 ulv2 

Out: ai a2 b lb2 clc2 wlw2 xlx2 ulu2 vlv2 
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Display 

code: 03 12 13 02 02 03 12 

Dis- 
play: ala2 alb2 blc2 clc2 wlw2 xlx2 ulu2 ulv2 . . . 

5 71. Movie to movie 49: 

In: ala2 alb2 blc2 clc2 vlw2 wlw2 xlx2 ulu2 ... 
Out: ala2 blb2 clc2 vlvl wlw2 xlx2 ulu2 ... 

Display 

code: 03 12 13 02 02 02 03 

10 Dis- 
play: ala2 alb2 blc2 clc2 vlvl wlw2 xlx2 ulu2 . . . 



72. Movie to movie 50: 
In: ala2 alb2 blc2 clc2 
Out: ala2 blb2 

15 Display 

code: 03 12 
Dis- 
play: ala2 alb2 blc2 

73. Movie to movie 51: 
20 In: ala2 alb2 blc2 clc2 

Out: ala2 blb2 

Display 

code: 03 12 

Dis- 

25 play: ala2 alb2 blc2 

74. Movie to movie 52: 
In: alb2 blc2 clc2 dld2 
Out : blb2 clc2 
Display 

30 code: 12 13 

Dis- 



ulv2 vlw2 wlw2 xlx2 . . . 
clc2 ulv2 vlvl wlw2 xlx2 . . . 

13 02 02 02 02 

Clc2 ulv2 vlvl wlw2 xlx2 . . . 

ulu2 ulv2 vlw2 wlw2 . . . 
clc2 ulu2 vlv2 wlw2 . . . 

13 . 03 12 13 

clc2 ulu2 ulv2 vlw2 wlw2 . . . 

xlx2 ulu2 ulv2 vlw2 . . . 
dld2 Xlx2 ulu2 vlv2 

02 02 03 12 
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play: alb2 blc2 clc2 dld2 xlx2 ulu2 ulv2 vlw2 

75. Movie to movie 53: 

In: alb2 blc2 clc2- dld2 wlw2 xlx2 ulu2 ulv2 
Out : blb2 clc2 dld2 wlw2 xlx2 ulu2 vlv2 

.5 Display 

Code: 12 13 02 02 02 03 12 

Dis- 
play: alb2 blc2 clc2 dld2 wlw2 xlx2 ulu2 ulv2 

76. Movie to movie 54: 
10 In: alb2 blc2 clc2 dld2 vlw2 wlw2 xlx2 ulu2 

Out : blb2 clc2 dld2 W2w2 wlw2 xlx2 ulu2 

Display 

COde: I 2 13 02 02 020 02 03 

Dis- 

15 play: a ib2 blc2 clc2 dld2 w2w2 wlw2 xlx2 ulu2 

77. Movie to movie 55: 

In: blc2 clc2 dld2 ulv2 vlw2 wlw2 xlx2 ulu2 

° Ut: ' clc2 dld2 ulv2 vlvl wlw2 xlx2 ulu2 ... ' 

Display 

20 COde: 13 .02 02 02 02 02 03 

Dis- 
play: blc2 C1C2 dld2 ulv2 vlvl wlw2 xlx2 ulu2 ... 

78. Movie to movie 56: 

In: blc2 clc2 dld2 ulu2 ulv2 vlw2 wlw2 xlx2 
25 Out: clc2 dld2 ulu2 vlv2 wlw2 xlx2 

Display 

■ code: 13 02 03 12 13 02 

Dis- 
play: blc2 clc2 dld2 ulu2 ulv2 vlw2 wlw2 xlx2 
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CLAIMS 

We claim: 



1. A video data stream analyzer comprising: 

a statistical analyzer having an input bus and 
5 an output bus wherein 

said statistical analyzer passes a video 
data stream on said input bus to said output bus 
and generates a plurality of parameters for each 
field in said video data stream; 
10 a reordering memory having an input bus coupled 

to said output bus of said statistical analyzer and 
an output line wherein each field in said video data 
stream is stored in said reordering memory; and 
a controller coupled to said statistical 
15 analyzer to receive said plurality of parameters and 

coupled to said reordering memory 

wherein said controller analyzes said 
plurality of parameters and selects one of said 
stored fields in said reordering memory for 
output onto said reordering memory output line. 



20 



2 . A video data stream analyzer as in Claim 1 
wherein said statistical analyzer further comprises: 

a first-in-first-out memory, hereinafter FIFO 
memory, having an input bus and an output bus 

wherein said FIFO memory input bus is. said 
statistical analyzer input bus; and 

said FIFO memory output bus is said 
statistical analyzer output bus . 



3 . A video data stream analyzer as in Claim 2 
3 0 wherein said FIFO memory further comprises a first field 
memory connected to said FIFO memory input bus . 
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4 . A video data stream analyzer as in Claim 3 
wherein said FIFO memory further comprises a second field 
memory connected to said first field memory and to said 
FIFO memory output bus so that said video data stream 

5 passes through said first field memory to said second 
field memory and onto said FIFO memory output bus. 

5. A video data stream analyzer as in Claim 2 
wherein said statistical analyzer further comprises: 

a differential accumulator connected to said 
FIFO memory wherein said differential accumulator 
processes said video data stream passing through said 
FIFO memory and generates a difference measure for 
each field in said video data stream. 

S. A video data stream analyzer as in Claim 5 
15 wherein said difference measure is for a strip of a field. 



10 



7. A video data stream analyzer as in Claim 



wherein said differential accumulator" f urtneV^comprLes - 

a strip activity differential accumulator having 
a first input bus connected to said FIFO input bus 
20 and a second input bus connected to said FIFO output 

bus ; 

a differential activity differential accumulator 
having a first input bus connected to said FIFO 
output bus and a second input bus connected between a 

flrSt f±eld memor y and * second field memory of said 
FIFO memory; 1 

a line delay memory having an input bus 
connected to said FIFO memory output bus and an 
output bus; and 

30 an iatra-activity differential accumulator 

having a first input bus connected to said FIFO 
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memory output bus and a second input bus connected to 
said line delay memory output bus. 

8 . A video data stream analyzer as in Claim 1 
wherein said controller further comprises a statistical 

5 processor coupled to receive said parameters from said 
statistical analyzer and a state machine coupled to said 
statistical processor. 

9 . A video data stream analyzer as in Claim 8 
wherein said state machine detects in real time a scene 

10 change in said video data stream. 

10. A video data stream analyzer as in Claim 8 
wherein said state machine detects in real time a repeated 
field in said video data stream. 



11. A video data stream analyzer as in Claim 8 

15 wherein .said state' machine detects in real time a change 
from a video sequence in said video data stream to a 
cartoon sequence . • 

12 . A video data stream analyzer as in Claim 8 
wherein said state machine detects in real time a change 

2 0 from a video sequence in said video data stream to a 
telecine sequence. 

13 . A video data stream analyzer as in Claim 8 
wherein said state machine detects in real time a change 
from a cartoon sequence in said video data stream to a 

25 video sequence. 



14 . A video data stream analyzer as in Claim .8 
wherein said state machine detects in real time a change 
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from a cartoon sequence in said video data stream to a ' 
telecine sequence. 

15. A video data stream analyzer as in Claim 8 

wherein said state machine detects in real time a change 

5 from a telecine sequence in said video data stream to a 
video sequence, 

16. A video data stream analyzer as in Claim 8 
wherein said state machine detects in real time a change 
from a telecine sequence in said video data stream to a 

10 cartoon sequence. 

17. a video data stream analyzer as in Claim 8 
further comprising a plurality of registers coupled to 
said controller. 

18. A video data stream analyzer as in Claim 17 

15 wherein a state of a bit in each of said plurality of 

registers represents a characteristic of a field in said 
reordering memory. 

19. A video data stream analyzer as in Claim 17 
wherein one register in said plurality of registers 

20 comprises a repeated field register. 

.20.. A video data stream analyzer as in Claim 19 
.wherein a bit in said repeated field register indicates 
^whether a primary criterion for. a field ; in said reordering 
memory is satisfied. 

25 21 " A v i^o data stream analyzer as in Claim 17 ' 

wherein one register in said plurality of registers 
comprises a scene cut register. 




WO 95/15659 PCTAJS94/13287 

- 155 - 

22. A video data stream analyzer as in Claim 21 
wherein a bit in said scene cut field register indicates 
whether a scene cut occurs at a field in said reordering 
memory . 

5 23 . A video data stream analyzer as in Claim 17 

wherein one register in said plurality of registers 
comprises a scan pattern register. 

24. A video data stream analyzer as in Claim 23 
wherein a bit in said scan pattern register indicates 
10 whether a field pair in said reordering memory is 
progressive or interlaced. 



25 . A video data stream analyzer as in Claim 8 
wherein said state machine includes a video state and 
further wherein said state machine remains in said video 

15 state so long as said input video data stream is a video 
sequence . 

26. A video data stream analyzer as in Claim 25 
wherein said state machine further comprises a cartoon 
state wherein said state machine enters said cartoon state 

20 upon detecting a sequence of fields in said input video 
data stream that have characteristics indicating a 
possible start of a cartoon sequence. 

27. A video data stream analyzer as in Claim 26 
wherein said state machine further comprises a status 

25 condition having a first state and a second state wherein 
upon said status condition having said first state, said 
state machine transitions, in response to a predetermined 
signal, from said video state to said cartoon state . 
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28. A video data. stream analyzer as in Claim 27 
wherein said controller further comprises a repeated field 
register and an odd-even compare register wherein said 
status condition is determined based on information stored 

S in said repeated field register and said odd-even compare 
register. 

29. A video data stream analyzer as in Claim 25 
wherein said state machine further comprises a first seek 
state wherein said state machine enters said first seek 

10 state upon said state machine detecting a sequence of 

fields in said input video data stream that indicates that 
a telecine sequence may be starting in said input video 
data stream. 

30. a video data stream analyzer as in Claim 29 
15 wherein said state machine further comprises a status 

condition having a first state and a second state wherein 
upon said status condition having said first state, said 
state machine transitions, in response to a. predetermined 
signal, from said video state to said first seek state. 

20 31. A video data stream analyzer as in Claim 3 0 

wherein said controller further comprises a repeated field 
register and an odd-even compare register wherein said 
first status condition is determined based on information 
stored in said repeated field register and said odd-even 

25 compare register. 

32. A video data stream analyzer as in Claim 30 
wherein upon said status condition having said first state 
and said state machine is in a cartoon state, said state 
machine transitions, in response to a predetermined 
3 0 signal, from said cartoon state to said first seek state 



I 
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33. A video data stream analyzer as in Claim 29 
wherein said state machine further comprises a movie state 
wherein said state machine enters said movie state upon 
said state machine detecting a sequence of fields in said 

5 input video data stream that indicates a telecine 
sequence . 

34. A video data stream analyzer as in Claim 33 
wherein said state machine further comprises a status 
condition having a first state and a second state wherein 

10 upon said status condition having said first state, said 
state machine transitions, in response to a predetermined 
signal, from said first seek state to said movie state; 
and upon said status condition having said second state, 
said state machine transitions from said status seek state 

15 to said video state. 

35. A video data stream analyzer as in Claim 34 
wherein said controller further comprises a repeated field 
register wherein said status condition is determined based 
on information stored in said repeated field register. 

20 36. A video data stream analyzer as in Claim 34 

wherein said controller further comprises a repeated field 
register and an odd-even compare register wherein said 
first status condition is determined based on information 
stored in said repeated field register and said odd-even - 

25 compare register. 

37. A video data stream analyzer as in Claim 33 
wherein said state machine further comprises a second seek 
state. 

38. A video data stream analyzer as in Claim 37 
3 0 wherein said state machine further comprises a status 
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condition having a first state and a second state wherein * 
upon said status condition having said first state, said 
state machine transitions, in response to a predetermined 
signal, from said movie state to said second seek state; 
5 and upon said status condition having said second state,' 
said state machine transitions, in response to said 
predetermined signal, to said video state. 

39. A video data stream analyzer as in Claim 3 8 
wherein said controller further comprises a repeated field 

10 register and an odd-even compare register wherein said 

status condition is determined based on information stored 
in said repeated field register and odd-even compare 
register. 

40. A video data stream analyzer as in Claim 3 8 
15 wherein said state machine further comprises a second 

status condition having a first state and a second state 
wherein upon said second status condition having said 
first state and. said state machine is in said second seek 
state, said state machine transitions, in response to a 
2 0 predetermined signal, from said second seek state to said 
movie state. 

41. a video data stream analyzer as in Claim l 
further comprising a side channel coupled to said 
controller wherein said controller generates a display . 
25 code on said side channel. 

42 . A video data stream analyzer as in Claim l 
further comprising a side channel coupled to said 
controller wherein said controller generates a output 
format code on said side channel. 
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43. A video data stream analyzer as in Claim! 
wherein said reordering memory further comprises : 

a first first-in-first-out memory, hereinafter 
FIFO memory, having an input bus connected to said 
5 statistical analyzer output bus and an output bus; 

a second first -in- first -out memory having an 
input bus connected to said statistical analyzer 
output bus and an output bus; 

a video random access memory connected to said 
10 output bus of said first FIFO memory and to said 

output bus of said second FIFO memory; 

a read-write circuit coupled to* said video 
random access memory and coupled to said controller; 
and 

15 a video random access memory controller coupled 

to said video random access memory and to said read- 
write circuit . 

44. A video data stream analyzer comprising: 

a first- in- first-out memory, hereinafter FIFO 
20 memory, having an input bus and an output bus; 

a reordering memory having an input bus coupled 
to said output bus of said FIFO memory and an output 
line 

wherein a video data stream on said input 
25 bus of said FIFO memory is passed .through said 

FIFO memory to said reordering memory; and 

each field in. said video data stream is 
stored in said reordering memory, 
a differential accumulator connected to said 
3 0 FIFO memory wherein said differential accumulator 

processes said video data stream passing through said 
FIFO memory and generates a difference measure for 
each field in said video data stream; and 
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a controller connected to said reordering memory 
and to said differential accumulator wherein 

said controller analyzes said . difference 
measures from said differential accumulator and 
5 selects one of said stored fields in said 

reordering memory for output onto said 
reordering memory output line. 

45. A video data stream analyzer as in Claim 44 
further comprising a side channel coupled to said 

10 controller wherein said controller generates a display 
code on said side channel . 

46. A video data stream analyzer as in Claim 44 
further comprising a side channel coupled to said 
controller wherein said controller generates a output 

15 format code on said side channel. 

47. a video data stream analyzer as in Claim 44 
wherein said controller further comprises a statistical 
processor and a state machine. 

48. A video data stream analyzer as in Claim 47 

20 wherein said state machine includes a plurality of states. 

49. A video data stream analyzer as in Claim 48 
wherein said state machine includes a plurality of status 

■^conditions and further wherein said status conditions are 
used by said state machine in real time to determine 
25 transitions between states in said plurality of states. 

50. A video data stream analyzer as in Claim 48 
wherein said plurality of states include a video state a 
cartoon state, a first seek state, a movie state, a second 
seek state, and a transfer state. 
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51. A video data stream analyzer as in Claim 44 
further comprising a plurality of registers coupled to 
said controller. 

52. A video data stream analyzer as in Claim 51 

5 wherein said plurality of registers includes a repeated 
field register, a scan pattern register, an odd-even 
compare register, and a scene cut register. 

53 . A video data stream analyzer as in Claim 44 
wherein said differential accumulator further comprises: 

10 a strip activity differential accumulator having 

a first input bus connected to said FIFO input bus 
and a second input bus connected to said FIFO output 
bus ; 

a differential activity differential accumulator 
15 having a first input bus connected to said FIFO input 

bus and a second input bus connected between a first 
field memory and a second field memory of said FIFO 
memory ; 

a line delay memory having an input bus 
20 connected to said FIFO memory output bus and an 

output bus ; and 

a intra-activity differential accumulator having 
a first input bus connected to said FIFO memory 
output bus and a second input bus connected to said 
25 line delay memory output bus. 

54 . A video data stream analyzer as in Claim 44 
wherein said reordering memory further comprises : 

a second first-in-first-out memory, hereinafter 
second FIFO memory, having an input bus connected to 
3 0 said first mentioned FIFO output bus and an output 

bus ; 



10 
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a third first-in-first-out memory having an 
input bus connected to. said first mentioned FIFO 
output bus and an output bus; 

a video random access memory connect to said 
output bus of said second FIFO memory and to said 
output bus of said third FIFO memory; 

a read-write circuit coupled to said video 
random access memory and to said controller; and 

a video random access memory controller coupled 
to said video random access memory and to said read 
write circuit. 



15 



20 



55. a method for preprocessing a stream of video 
data, that can include video data from any one of a 
Plurality of video data types, comprising the steps of: 
generating a plurality of parameters that 
characterize a field for each field in said stream of 
video data; and 

identifying a video data type in said plurality 
of video data types in said stream of video data 
using said plurality of parameters. 

56 . The method of Claim 55 wherein said steps are 
performed in real time. 

- S7. The method of claim 55 further comprising the 
■ ... step of : 

25 - i using one of said plurality of parameters to 

identify a scene cut in said stream of video data. 

SB' The method of Claim 57 wherein said steps are 
performed in real time. ' 
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59. The method of Claim 55 further comprising the 
step of : 

passing each field in said stream of video data 
through a first memory. 

5 60. The method of Claim 5 9 wherein said generating 

step further comprises : 

generating a measure of activity for each field 
using an accumulator connected to said first memory. 

61. The method of Claim 60 wherein said accumulator 
10 is a differential accumulator. 

62. The method of Claim 6 0 wherein said measure of 
activity is a differential activity between a strip of a 
first field having a parity and a strip of a second field 
of said parity and so - said measure of activity is a strip 

15 activity for said first field. 

63. The method of Claim 62 wherein said generating 
step further comprises : 

storing each of said strip activities for a 
field. 

20. 64. The method of Claim 63 wherein said generating 

step further comprises : 

summing said stored strip activities for a field 
to generate a field activity for said field. 

65. The method of Claim 64 wherein said 
25 identification step further comprises: 

comparing said field activity of said field with 
a first threshold; and 
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identifying said field as a repeated field if 
said field activity of said field is i ess than said 
first threshold and said field has a first parity. 

66. The method of Claim 65 wherein said first parity 
S is even parity. ^ y 

67. The method of Claim 55 wherein said 
identification step further comprises: 

comparing each of said stored strip activities 
of sa ld field with a second threshold; and 

identifying said field as a repeated field if a 
predetermined number of said strip activities of said 
field are less than said second threshold and said 
rield has said first parity. 

68. The method of Claim 67 wherein said 
15 identification step further comprises: 

repeating said field activity comparison and 
saxd strip activity comparison steps for a second . 
field of said first parity if Vi> said field has a 
second parity; (ii) said field Qf ^ 

" l6SS than said ^st threshold; and (iii) a 
predetermined number of said strip activities of said 
field are less than said second threshold; and 

identifying said field as a repeated field if 
(i) said field activity of said second field is 
greater than said first threshold; or (ii) a 
predetermined number of said strip activities of said 
second f.eld are greater than said second threshold. 

69. The method of Claim 68 wherein said 
identification step further comprises: 
30 comparing a measure of field activity of said 

field with a third threshold if (i) said field 
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activity of said second field is less than said first 
threshold; and (ii) a predetermined number of said 
strip activities of said second field are less than 
said" second threshold; and 
5 identifying said field as a repeated field if 

said measure of field activity of said field is less 
than said third threshold. 

70. The method of Claim 55 wherein said 
identification step further comprises : 
10 determining if a primary criterion is true for a 

first field in a frame; and 

identifying the first field as an unrepeated 
field upon determining that the primary criterion is 
not true. 

15 71. The method of Claim 70 wherein said 

identification step further comprises: 

determining if the primary criterion is true for 
another field in said frame upon determining that the 
primary criterion is true for the first field. 

20 72. The method of Claim 71 wherein said 

identification step further comprises: 

identifying the first field as a repeated field 
upon determining that the primary criterion is not 
true for another field in said frame. 

25 73. The method of Claim 72 wherein said 

identification step further comprises: 

determining if a secondary criterion is true for 
the first field upon determining that the primary, 
criterion is true for another field in said frame. 



10 
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74. The method of Claim 73 wherein said 
identification step further comprises: 

identifying the first field as an unrepeated 
field upon determining that the secondary criterion 
is not true. 

75. The method of Claim 73 wherein said 
identification step further comprises: 

determining if a secondary criterion is true for 
another field in said frame upon determining that the 
secondary criterion is true for the first field. 

76. The method of Claim 75 wherein said 
identification step further comprises: 

identifying the first field as a repeated field 
upon determining that the secondary criterion is not 
true for another field in said frame. 

77. The method of Claim 55 wherein said 
identification step further comprises: 

analyzing a pattern of fields identified as 
repeated fields to identify one of a cartoon sequence 
and a telecine sequence in said stream of data. 

78 . The method of Claim 77 further comprising the 
step of: 

dropping repeated fields from said stream of 
■ v. ; data based UP°* identification of one of a cartoon 
2 5 sequence and a telecine sequence. 

79. The method of Claim 78 wherein said steps of 
saxd method are performed in real time. 

80. The method of Claim 57 further comprising the 
step of: 
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passing each field in said stream of video data 
through a first memory. 



81. The method of Claim 80 wherein said generating 



step further comprises : 
5 using an accumulator connected to said first 

memory to generate a measure of activity. 

82. The method of Claim 81 wherein said accumulator 
is a differential accumulator. 

83. The method of Claim 82 wherein said measure of 
10 activity is a differential activity between a strip of a 

first field having a parity and a second field of said 
parity and so said measure of activity and so is a strip 
activity for said first field. 

15 84 . The method of Claim 83 wherein said generating 

step further comprises: 



85. The method of Claim 84 wherein said generating 
20 step further comprises: 

summing said stored strip activities for a field 
to generate a field activity for said field. 



storing each of said strip activities for a 



field. 



86. The method of Claim 85 wherein said 



identification step further comprises: 



25 



identifying a scene cut between a first field 
and a second field upon said field activity for said 
first field satisfying a predetermined condition. 



87. The method of Claim 86 wherein said 
predetermined condition is that said field activity be 
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greater than a predetermined minimum value and greater 
than a scene cut threshold. 

B8. The method of Claim 87 wherein said scene cut 
threshold if a running average of field activities for 
5 fields having a parity that is the same as a parity of 
said first field. 

89. The method of Claim 8 6 further comprising the 
step of : 

resetting thresholds used in a repeat field 
10 detection process upon identification of a scene cut. 

90 . A method for processing telecine sequences and 
cartoon sequences in a video data stream comprising the 

steps of: 

monitorin 9 each f ie ld in said video data stream 
to detect repeated fields wherein upon detection of a 
repeated field, the repeated field is flagged; 

analyzing a pattern formed by said fields 
flagged as repeated to identify whether the repeated 
fields are in a cartoon sequence or a telecine 
20 sequence ; 

dropping repeated fields from said video data 
stream based upon identification of the repeated 
field as being in one of said cartoon sequence and 
said telecine sequence. 

25 9i. The method of claim 9Q wherein sa . d sfceps 

performed in real time. 

92. A method for detecting a scene cut in a video 
data stream comprising the steps of: 

measuring a cumulative difference in activity 
30 between a first field having a parity in said video 
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data stream and a second field of said parity in said 
video data stream; and 

identifying a scene cut between said first and 
second fields upon said cumulative difference 
5 satisfying a predetermined condition. 

93 . The method of Claim 92 wherein said steps are 
performed in real time. 

94 . The method of Claim 92 wherein said step of 
measuring a cumulative difference in activity further 

10 comprises : 

measuring a field activity between said first 
and second fields of said parity. 

95 . The method of Claim 94 wherein said step of 
measuring a field activity further comprises : 

15 measuring a plurality of strip activities 

between first and second fields of said parity. 

96. The method of Claim 95 wherein said step of 
measuring a field activity further comprises: 

summing said plurality of strip activities to 
20 form said field activity. 

97. The method of Claim 95 wherein said step of 
measuring a plurality of strip activities further 
comprises : 

dividing a field into an integer number of 

25 strips. 



98. The method of Claim 97 wherein said step of 
measuring a plurality of strip activities further 
comprises : 
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measuring the strip activity for each of said' 
integer number of strips so that said plurality of 
strip activities is said integer number of strip 
activities. 

■5 99. The method of Claim 94 wherein said step of 

measuring a field activity further comprises: 

passing each field in said video data stream 
through a first field memory having an input bus and 
an output bus . 



10 



100. The method of Claim 99 wherein said step of 
measuring a field activity further comprises : 

passing each field in said video data stream 
through a second field memory having an output bus 
and an input bus connected to the output bus of said 
15 first field memory. 

101. The method of Claim 100 wherein said step of 
measuring a field activity further comprises: 

accumulating pixel differences for a plurality 
of lines in successive fields of the same parity 
20 using a difference accumulator connected to said 

input bus of said first field memory and to said 
output bus of said second field memory. 

102. The method of Claim 101 wherein said step of 
.measuring a field activity further comprises: 

accumulating said accumulated pixel differences 
for a field and thereby measure said field activity. 

103 . The method of Claim 94 wherein said step of 
measuring a cumulative difference in activity further 

comprises : 
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generating a running average of said field 
activities . 

104. The method of Claim 103 wherein said running 
average is updated for a field only if a predetermined 

5 condition is true. 

105. The method of Claim 104 wherein said 
predetermined condition is that the absence of a sudden 
deviation between the field activity of the field and said 
running average . 

10 106. The method of Claim 103 wherein said running 

average is generated using a recursive filter. 

107. The method of Claim 103 wherein said 
predetermined condition further comprises: 

comparing said field activity with a scene cut 
15 threshold. 

108. The method of Claim 107 wherein said 
predetermined condition further comprises: 

comparing said field activity with a minimum 
value threshold wherein said field activity 
20 satisfies said predetermined condition if said 

field activity is greater than both said scene 
cut threshold and said minimum value threshold. 

109. The method of Claim 92 wherein said 
predetermined condition is that said cumulative difference 

25 be greater than a predetermined minimum value and greater 
than a scene cut threshold. 



110 . A method for detecting a scene cut in a video 
data stream having a plurality of frames wherein each 
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frame has a first field of a first parity and a second 
field of a second parity, said method comprising the steps 
of: 

dividing each frame into a plurality of strips ; 
5 forming a strip activity for each of said strips 

wherein said strip activity is a sum of absolute 
differences between a pixel in a first field of a 
first parity and a pixel of a second field of said 
first parity in said video data stream; 
10 summing said strip activities for said first 

field to form a field activity; 

identifying a . scene cut between said first and 
second fields upon said field activity satisfying a 
predetermined condition. 



15 



111. A method for detecting a repeated field in a 
video data stream having a plurality of frames wherein 
each frame includes a field of .a first parity and a field 
of a second parity, said method comprising the steps of: 
comparing a field activity of a field with a 
2 0 first threshold; and 

identifying said field as a repeated field if 
said field activity of said field is less than said 
first threshold and said field has said second 
parity. 



25 



30 



112. The method of Claim 111 wherein said steps are 
performed in real time. 

113. The method of Claim ill further comprising: ; 
comparing a strip activity of each strip of said 

field with a second threshold; and 

identifying said field as a repeated field if a 
predetermined number of said strip activities of said 
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field are less than said second threshold and said 
field has said second parity. 

114. The method of Claim 113 further comprising: 
repeating said field activity and strip activity 

5 comparison steps for a second field of said second 

parity if (i) said field has said first parity; (ii) 
said field activity of said field is less than said 
first threshold; and (iii) a predetermined number of 
said strip activities of said field are less than 

10 said second threshold; and 

identifying said field as a repeated field if 
(i) said field activity of said second field is 
greater than said first threshold; or (ii) a 
predetermined number of said strip activities of said 

15 second field are greater than said second threshold. 

115. The method of Claim 113 further comprising: 
repeating said field activity and strip activity 

comparison steps for a second field of said second 
parity if (i) said field has said first parity; (ii) 

20 said field activity of said field is less than said 

first threshold; and (iii) a predetermined number of 
said strip activities of said field are less than 
said threshold; 

comparing a measure of field activity of said 

25 field with a third threshold if (i) said field 

activity of said second field is less than said first 
threshold; and (ii) a predetermined number of said 
strip activities of said second field are less than 
said second threshold; and 

30 identifying said field as a repeated field if 

said measure of field activity of said field is less 
than said third threshold. 
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116. The method of claim 113 wherein said strip 
activity is a sum, for all pixels in a strip, of an 
absolute difference between a pixel in a first field of 
one parity and a pixel of a second field of said one 
5 parity. 



117. The method of Claim 116 wherein said field 
activity is a sum of each strip activity of a field. 

118 . The method of Claim ill further comprising the 
step of initializing said first threshold to four times 

10 the number of pixels in said field upon power-up. 

119. The method of Claim 113 further comprising the 
step of initializing said first threshold to six times the 
number of pixels in a strip of said field upon detection 
of a scene cut in said video data stream. 

IS 120. The method of any one of Claims 111, 113, 114 or 

115 further comprising the step of : 

updating said first threshold upon identifying 
said field as a repeated field. 

121. The method of Claim 120 wherein said updating 
20 step further comprising: 

comparing said field activity with a stored 
field activity,- 

replacing said stored field activity with said 
field activity if said field activity is greater than 
^ . said stored field activity; and 

J - defining said first threshold in terms of said 

stored field activity. 

122. The method of Claim 120 further comprising the 

step of: ' . 
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updating said second threshold after updating 
said first threshold. 

123. The method of Claim 122 wherein said updating 
step further comprises : 

5 defining said second threshold in terms of said 

first threshold. 

124 . A method for detecting a repeated field in a 
video data stream comprising the steps of : 

measuring a strip activity between a strip in a 
10 first field of a first parity and a strip of a second 

field of said first parity in said video data steam 
for a plurality of strips thereby obtaining a 
plurality of strip activities for said first field; 

generating a field activity for said first field 
15 using said plurality of strip activities; 

comparing said field activity to a first 
threshold; 

comparing each of said plurality of strip 
activities to a second threshold; and 

20 identifying said first field as a repeated field 

if (i) if said first parity is an even parity; (ii) 
said field activity for said first field is less than 
said first threshold; and (iii) a predetermined 
number of said strip activities of said first field 

25 are less than said second threshold. 

125. The method of Claim 124 further comprising: 
performing the following steps if (i) said first 

parity is an odd parity; (ii) said field activity for 
said first field is less than said first threshold; 
3 0 and (iii) a predetermined number of said strip 

activities of said first field are less than said 
second threshold; 
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measuring a strip activity between a strip 
in a first field of an even parity and a strip 
of a second field of said even parity in said 
video data stream for a plurality of strips 
thereby obtaining a plurality of strip 
activities for said first field of said even 
parity; 

generating a field activity for said first 
field of said even parity using said plurality 
of strip activities for said first field of said 
even parity; 

comparing said field activity for said 
first field of said even parity to said first 
threshold; 

comparing each of said plurality of strip 
activities for said first field of said even 
parity to said second threshold; and 

identifying said first field of said first 
parity as a repeated field if (i) sai d field 
activity for said first field of said even 
parity is greater than said first threshold; . or 
Cii) a predetermined number of said strip 
activities for said first field of said even 
parity are greater than said second threshold. 

126. The method of Claim 124 further comprising: 

performing the following steps if (i) sa id first 
parity is an odd parity; (ii) said field activity for 
said first field is less than said first threshold; 
and (ni) a predetermined number of said strip 
activities of said first field are less than said 
second threshold; 

measuring a strip activity between a strip 
in a first field of an even parity and a strip 
of a second field of said even parity in said 
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video data stream for a plurality of strips 
thereby obtaining a plurality of strip 
activities for said first field of said even 
parity; 

5 generating a field activity for said first 

field of said even parity using said plurality 
of strip activities for said first field of said 
even parity; 

comparing said field activity for said 
10 first field of said even parity to said first 

threshold ,- 

comparing each of said plurality of strip 
activities for said first field of said even 
parity to said second threshold 

15 comparing a measure of field activity of 

said field with a third threshold if (i) said 
field activity of said first field of even 
parity is less than said first threshold; and 
(ii) a predetermined number of said strip 

2 0 activities of said first field of even parity 

are less than said second threshold; and 

identifying said field as a repeated . field 
if said measure of field activity of said field 
is less than a third threshold. 

25 127. The method of any one Claims 124, 125 or 126 

wherein said strip activity is a sum, for all pixels in a 
strip, of an absolute difference between a pixel in a 
first field of one parity and a pixel of a second field of 
said one parity. 

30 128. The method of any one of Claims 124, 125 or 126 

further comprising the step of : 

updating said first threshold upon identifying 
said first field as a repeated field. 
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129. The method of Claim 128 wherein said updating' 
step further comprising: 

comparing said field activity of said first 
field with a stored field activity; 
5 replacing said stored field activity with said 

field activity of said first field if said field 
activity of said first field is greater than said 
stored field activity; and 

defining said first threshold in terms of said 
10 stored field activity. 

130. The method of Claim 129 further comprising the 
step of: 

updating said second threshold after updating 
said first threshold. 

15 131. The method of Claim 130 wherein said updating 

step further comprises: 

defining said second threshold in terms of said 
first threshold. 

132. A method for detecting repeated fields in a 
2 0 sequence of frames of video data where each frame has a 
field of a first parity and a field of a second parity: 

determining if a primary criterion is true for a 
first field in a frame; and 

identifying the first field as an unrepeated 

25 . : field u P° n determining that the primary criterion is 
not true. 

133. The method of Claim 132 further comprising the 
step of: 

determining if the primary criterion is true for 
3 0 the other field in said frame upon determining that 

the primary criterion is true for the first field. 
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134. The method of Claim 133 further comprising the 
step of : 

identifying the first field as a repeated field 
upon determining that the primary criterion is not 
5 true for the other field in said frame. 

13 5. The method of Claim 134 further comprising the 
step, of : 

determining if a secondary criterion is true for 
the first field upon determining that the primary 
10 criterion is true for the other field in said frame. 

136. The method of Claim 135 further comprising the 
step of : 

identifying the first field as an unrepeated 
field upon determining that the secondary criterion 
15 is not true. 

137. The method of Claim 135 further comprising the 
step of : 

determining if a secondary criterion is true for 
the other field in said frame upon determining that 
20 the secondary criterion is true for the first field. 

138. The method of Claim 137 further comprising the 
step of : 

identifying the first field as a repeated field 
upon determining that the secondary criterion is not 

2 5 true for the other field in said frame. 

139. The method of Claim 13 2 wherein the primary 
criterion is true if a predetermined number of strip 
activities for the first field do not exceed a first 
threshold, and if a field activity for. the first field 

3 0 does not exceed a second threshold. 



WO 95/15659 



{ , 

PCT/US94/13287 



- 180 - 

140. The method of Claim 139 wherein the step of 
determining if the primary criterion is true further 
comprises : 

measuring a strip activity between a strip 
5 in a first field of a given parity and a strip 

of a second field of said given parity in said 
video data stream for a plurality of strips 
thereby obtaining a plurality of strip 
activities for said first field of said first 
10 parity; 

generating a field activity for said first 
field of said given parity using said plurality 
of strip activities for said first field of said 
given parity. 

15 141. The method of Claim 13 6 ' wherein the secondary 

criterion is true if a measure of a field activity for the 
first field does not exceed a third threshold. 

142. A method for detecting repeated fields in a 
video data sequence of frames having a field of a first 
2 0 parity and a field of a second parity comprising the steps 
of : 

measuring a differential parameter for pixels in 
a line of a first field of a given parity and pixels 
in a line of a second field of said given parity; and 
5 comparing said differential parameter with a 

threshold to detect a repeated field. 

143 . The method of Claim 142 wherein said steps are 
performed in real time. 

144. A real time method for detecting a telecine 

0 sequence in a video data stream of frames having a field 
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of a first parity followed by a field of a second parity, 
said method comprising the steps of : 

monitoring in real time each field in said video 
data stream to detect repeated fields and flagging 
5 each repeated field; and 

analyzing in real time a pattern formed by said 
fields flagged as repeated to identify whether the 
repeated- fields are in a telecine sequence. 

145. The method of Claim 144 wherein said analyzing 
10 step further comprises determining whether every fifth 
field is a repeated field. 



146. The method of Claim 144 wherein said monitoring 
step further comprises measuring a differential parameter 
for pixels in a line of a first field of a given parity 

15 and pixels in a line of a second field of said given 
parity. 

147. The method of Claim 146 wherein a repeated field 
is detected by comparison of . said differential parameter 
with a threshold. 

20 14 8. The method of Claim 144 wherein said analyzing 

step further comprises : 

upon detecting that in a frame, a first field of 
said first parity is flagged as a repeated field and 
a first field of said second parity is not flagged as 
25 a repeated field, determining whether a second field 

of said second parity, that is a fifth field after 
said first field of said first parity, is a repeated 
field; 

identifying five frames starting with said frame 
3 0 with a repeated first field of said first parity as a 
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telecine sequence if said second field of said second 
parity is flagged as a repeated field. 

149. The method of Claim 145 wherein said analyzing 
step further comprises the step of: 

upon determining that said second field of said 
second parity is flagged as a repeated field, 
determining (i) whether a second field of said first 
parity, that is a fifth field after said second field 
of said second parity field, is a repeated field; and 
(ii) a third field of said second parity, that is 
immediately after said second field of said first 
parity, is not a repeated field. 

150. The method of Claim 146 wherein said analyzing 
step further comprises the step of: 

upon determining that (i) a second field of said 
first parity, that is a fifth field after said second 
field of said second parity field, is flagged as a 
repeated field; and (ii) a third field of said second 
parity, that is immediately after said second field 
of said first parity, is not flagged as a repeated 
field; determining whether a third field of said, 
second parity, that is a fifth field after said 
second field of said first parity, is a repeated 
field. 



25 



151. The method of Claim 150 wherein said analyzing 
step further comprises the step of: 

identifying five frames, starting with a frame 
containing said second field of said first parity, as 
a telecine sequence if said third field of said 
30 second parity, that is a fifth field after said 

second field of said first parity, is flagged as a 
repeated field. 
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152 . A method for detecting a cartoon sequence in a 
video data stream of frames having a field of a first 
parity followed by a field of a second parity, said method 
comprising the steps of: 

5 monitoring each field in said video data stream 

to detect repeated fields and flagging each repeated 
field; and 

analyzing a pattern formed by said fields 
flagged as repeated to identify whether the repeated 
10 fields are in a cartoon sequence . 

153 . The method of Claim 152 wherein said steps are 
performed in real time. 

154. The method of Claim 152 wherein said analyzing 
step further comprises determining whether the two fields - 

15 in every other frame of said sequence are flagged as 
repeated fields. 

155 . The method of Claim 152 wherein said monitoring 
step further comprises measuring a differential parameter 
for pixels in a line of a first field of a given parity 

20 and pixels in a line of a second field of said given 
parity. 

156. The method of Claim 155 wherein a repeated field 
is detected by comparison of said differential parameter 
with a threshold. 

25 157. A method for improving encoding efficiency of a 

video' data stream, said method comprising the steps of: 
analyzing each field in said video data stream 
to detect repeated fields ; and 

dropping each detected repeated field from said 
3 0 video data stream to generate a modified video data 
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stream thereby eliminating encoding of redundant 
information and so improving encoding efficiency. 

158. The method. of Claim 157 wherein said steps are 
performed in real time. 

15 9. A method as in Claim 157 further comprising the 
step of : 

generating a display code for each frame in said 
modified video data stream wherein said display code 
indicates a display combination for fields in said 
10 modified video data stream to recreate said video 

data stream. 

160. A method for improving encoding efficiency of a 
video data stream, said method comprising the steps of: 
analyzing each frame in said video data stream 
15 to detect frames that include a mid-frame, edit; 

dropping a first field in a frame in which a 
mid- frame edit was detected; and 

replacing said first field with a second field 
in said frame in which a mid-f rame . edit was detected 
20 thereby eliminating an abrupt change in activity with 

said frame and so improving encoding efficiency. 

161. The method of Claim 160 wherein said steps are 
.performed in real time. 

162. A method for introducing dead times in a video 
25 data stream to a video encoder comprising the steps of: 

storing for each frame in said video data stream 
a field of a first parity and a field of a second 
parity; 
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passing selected stored fields to said video 
encoder in response to a first output field selection 
code ; and 

passing no stored fields to said video encoder 
5 in response to a second output field selection code 

wherein said second output field selection code is 
used to introduce a dead time in said video data 
stream. 

163. The method of Claim 162 wherein said steps are 
10 performed in real time. 

164. The method of Claim 162 wherein in response to 
said first output field selection code, one of said stored 
fields of a first parity and then one of said stored 
fields of a second parity are passed to said video 

15 encoder. 

165. The method of Claim 162 wherein in response to 
said first output field selection code, one of said stored 
fields of a second parity and then one of said stored 
fields of a first parity are passed to said video encoder. 

20 166. The method of Claim 162 wherein in response to 

said first output field selection code, one of said stored 
fields of a first parity is passed to said video encoder 
as both fields of a frame. 

167. The method of Claim 162 wherein in response to 
25 said first output field selection code, one of said stored 

fields of a second parity is passed to said video encoder 
as both fields of a frame. 

168. A method for scan pattern analysis of a video 
data stream comprising the steps of: 
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generating for each field pair in said video 
data stream a plurality of scan pattern activity 
indices; and 

analyzing said plurality of scan pattern 
5 activity indices to identify each field pair as one 

of interlaced and progressive. 

169. The method of Claim 168 wherein said steps are 
performed in real time. 

170. The method of Claim 168 wherein said scan 

0 pattern activity indices comprise inter- intra indices. 

171. The method of Claim 168 wherein said generating 
step further comprises generating a scan pattern activity 
index for each strip in a field and thereby generating 
said plurality of scan pattern indices. 



172. The method of Claim 171 wherein said analysis 
step further comprises : 

comparing each of said scan pattern activity 
indices to a scan pattern threshold. 

173 . The method of Claim 171 wherein said analysis 
step further comprises: 

setting a scan pattern activity index to a first 
predetermined value upon determining that a 
differentiated activity metric for said strip is less 
than a strip activity threshold. 

174. The method of Claim 172 wherein said analysis 
step further comprises : 

identifying said field pair as interlaced if a 
predetermined number of said scan pattern activity 
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indices are greater than said scan pattern threshold, 
and otherwise as progressive. 

175 . A real time method for scan pattern analysis of 
a video data stream having fields comprising the steps of : 
5 generating, for each of a plurality of strips in 

a first field and each of a plurality of strips in a 
second field, a differential strip activity index DAi 
between strip i in said first field and strip i in 
said second field wherein said first and second 
10 fields are a field pair; 

generating an intra strip activity IAli for each 
strip i in said first field and an intra strip 
activity IA2i for each strip i in said second field; 
forming an inter- intra scan pattern index for 
15 said first field for each of at least m of said 

strips wherein inter- intra scan pattern index for 
strip i is the ratio of differential index DAi to the 
sum of intra strip activity IAli and intra strip 
activity IA2i; 

20 identifying said field pair as interlaced if n 

of said inter- intra scan pattern indices are greater 
than unity and as progressive otherwise wherein m is 
equal to or greater than n. 
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